[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:07:01 dist-test-slave-nj21 krb5kdc[25318](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:07:01 dist-test-slave-nj21 krb5kdc[25318](info): set up 2 sockets
Aug 09 15:07:01 dist-test-slave-nj21 krb5kdc[25318](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 15:07:02.734853 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.329s user 0.000s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:07:02 dist-test-slave-nj21 krb5kdc[25318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:07:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:07:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
I20260809 15:07:02.903491 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:34303
--webserver_interface=127.24.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:41975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:34303
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:03.240127 25334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:03.240410 25334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:03.240549 25334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:03.247725 25334 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:07:03.247852 25334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:03.247931 25334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:03.248013 25334 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:07:03.248072 25334 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:07:03.257858 25334 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:41975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:34303
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:34303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:03.259230 25334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:03.260604 25334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:03.272581 25342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:03.272625 25346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:03.272668 25343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:03.273770 25334 server_base.cc:1034] running on GCE node
I20260809 15:07:03.274597 25334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:03.276170 25334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:03.277390 25334 hybrid_clock.cc:648] HybridClock initialized: now 1786288023277354 us; error 103 us; skew 500 ppm
I20260809 15:07:03.277772 25334 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:03 dist-test-slave-nj21 krb5kdc[25318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288023, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:03.281064 25334 init.cc:377] Logged in from keytab as kudu/127.24.183.254@KRBTEST.COM (short username kudu)
I20260809 15:07:03.282585 25334 webserver.cc:460] Webserver started at http://127.24.183.254:42667/ using document root <none> and password file <none>
I20260809 15:07:03.283290 25334 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:03.283391 25334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:03.283699 25334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:03.287003 25334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "a221e1c18a4d466aad134969aef4f1dc"
format_stamp: "Formatted at 2026-08-09 15:07:03 on dist-test-slave-nj21"
server_key: "70e79a8dbcf7e1cbcf9816959d87b71e"
server_key_iv: "a3c887476b99a256ed0065090a1eacd1"
server_key_version: "encryptionkey@0"
I20260809 15:07:03.287802 25334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "a221e1c18a4d466aad134969aef4f1dc"
format_stamp: "Formatted at 2026-08-09 15:07:03 on dist-test-slave-nj21"
server_key: "70e79a8dbcf7e1cbcf9816959d87b71e"
server_key_iv: "a3c887476b99a256ed0065090a1eacd1"
server_key_version: "encryptionkey@0"
I20260809 15:07:03.293154 25334 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 15:07:03.296667 25356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:03.297955 25334 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 15:07:03.298115 25334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "a221e1c18a4d466aad134969aef4f1dc"
format_stamp: "Formatted at 2026-08-09 15:07:03 on dist-test-slave-nj21"
server_key: "70e79a8dbcf7e1cbcf9816959d87b71e"
server_key_iv: "a3c887476b99a256ed0065090a1eacd1"
server_key_version: "encryptionkey@0"
I20260809 15:07:03.298285 25334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:03.314390 25334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:03.318583 25334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:03.318857 25334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:03.336566 25334 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:34303
I20260809 15:07:03.336588 25438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:34303 every 8 connection(s)
I20260809 15:07:03.337929 25334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:07:03.341989 25439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:03.347576 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 25334
I20260809 15:07:03.347900 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:07:03.349813 25311 external_mini_cluster.cc:1428] Setting key 5acdb0a796ddcbe1e5b23cbfb7ad9d34
I20260809 15:07:03.352440 25439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a221e1c18a4d466aad134969aef4f1dc: Bootstrap starting.
I20260809 15:07:03.356122 25439 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a221e1c18a4d466aad134969aef4f1dc: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:03.357123 25439 log.cc:824] T 00000000000000000000000000000000 P a221e1c18a4d466aad134969aef4f1dc: Log is configured to *not* fsync() on all Append() calls
I20260809 15:07:03.359794 25439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a221e1c18a4d466aad134969aef4f1dc: No bootstrap required, opened a new log
I20260809 15:07:03.368254 25439 raft_consensus.cc:348] T 00000000000000000000000000000000 P a221e1c18a4d466aad134969aef4f1dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a221e1c18a4d466aad134969aef4f1dc" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 34303 } }
I20260809 15:07:03.368705 25439 raft_consensus.cc:374] T 00000000000000000000000000000000 P a221e1c18a4d466aad134969aef4f1dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:03.368822 25439 raft_consensus.cc:729] T 00000000000000000000000000000000 P a221e1c18a4d466aad134969aef4f1dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a221e1c18a4d466aad134969aef4f1dc, State: Initialized, Role: FOLLOWER
I20260809 15:07:03.369415 25439 consensus_queue.cc:260] T 00000000000000000000000000000000 P a221e1c18a4d466aad134969aef4f1dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a221e1c18a4d466aad134969aef4f1dc" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 34303 } }
I20260809 15:07:03.369642 25439 raft_consensus.cc:388] T 00000000000000000000000000000000 P a221e1c18a4d466aad134969aef4f1dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:07:03.369772 25439 raft_consensus.cc:482] T 00000000000000000000000000000000 P a221e1c18a4d466aad134969aef4f1dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:07:03.369941 25439 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a221e1c18a4d466aad134969aef4f1dc [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:03.372480 25439 raft_consensus.cc:504] T 00000000000000000000000000000000 P a221e1c18a4d466aad134969aef4f1dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a221e1c18a4d466aad134969aef4f1dc" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 34303 } }
I20260809 15:07:03.373158 25439 leader_election.cc:302] T 00000000000000000000000000000000 P a221e1c18a4d466aad134969aef4f1dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a221e1c18a4d466aad134969aef4f1dc; no voters:
I20260809 15:07:03.373896 25439 leader_election.cc:288] T 00000000000000000000000000000000 P a221e1c18a4d466aad134969aef4f1dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:07:03.374064 25444 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a221e1c18a4d466aad134969aef4f1dc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:07:03.375093 25444 raft_consensus.cc:686] T 00000000000000000000000000000000 P a221e1c18a4d466aad134969aef4f1dc [term 1 LEADER]: Becoming Leader. State: Replica: a221e1c18a4d466aad134969aef4f1dc, State: Running, Role: LEADER
I20260809 15:07:03.375777 25444 consensus_queue.cc:237] T 00000000000000000000000000000000 P a221e1c18a4d466aad134969aef4f1dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a221e1c18a4d466aad134969aef4f1dc" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 34303 } }
I20260809 15:07:03.376194 25439 sys_catalog.cc:564] T 00000000000000000000000000000000 P a221e1c18a4d466aad134969aef4f1dc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:07:03.379076 25445 sys_catalog.cc:455] T 00000000000000000000000000000000 P a221e1c18a4d466aad134969aef4f1dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a221e1c18a4d466aad134969aef4f1dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a221e1c18a4d466aad134969aef4f1dc" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 34303 } } }
I20260809 15:07:03.379511 25445 sys_catalog.cc:458] T 00000000000000000000000000000000 P a221e1c18a4d466aad134969aef4f1dc [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:03.379961 25446 sys_catalog.cc:455] T 00000000000000000000000000000000 P a221e1c18a4d466aad134969aef4f1dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader a221e1c18a4d466aad134969aef4f1dc. Latest consensus state: current_term: 1 leader_uuid: "a221e1c18a4d466aad134969aef4f1dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a221e1c18a4d466aad134969aef4f1dc" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 34303 } } }
I20260809 15:07:03.380254 25446 sys_catalog.cc:458] T 00000000000000000000000000000000 P a221e1c18a4d466aad134969aef4f1dc [sys.catalog]: This master's current role is: LEADER
Aug 09 15:07:03 dist-test-slave-nj21 krb5kdc[25318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:03.390215 25450 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:07:03.396420 25450 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:07:03.398641 25441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:03.353825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45668 (local address 127.24.183.254:34303)
0809 15:07:03.354867 (+ 1042us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:03.354885 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:03.357171 (+ 2286us) server_negotiation.cc:407] Connection header received
0809 15:07:03.367615 (+ 10444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:03.367644 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:03.368098 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:03.368506 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:03.371483 (+ 2977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:03.373536 (+ 2053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:03.375997 (+ 2461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:03.377092 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:03.377291 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:03.385571 (+ 8280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:03.385596 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:03.385623 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:03.385682 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:03.390465 (+ 4783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:03.391550 (+ 1085us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:03.391560 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:03.391570 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:03.391873 (+ 303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:03.392521 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:03.392526 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:03.392529 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:03.393385 (+ 856us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:03.396390 (+ 3005us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:03.396665 (+ 275us) server_negotiation.cc:299] Negotiation successful
0809 15:07:03.397142 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":753,"thread_start_us":417,"threads_started":1}
I20260809 15:07:03.413772 25450 catalog_manager.cc:1347] Generated new cluster ID: 0f3aa44a1e58412ab4df482f43b591f1
I20260809 15:07:03.413884 25450 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:07:03.425346 25450 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:07:03.427057 25450 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:07:03.432857 25450 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a221e1c18a4d466aad134969aef4f1dc: Generated new TSK 0
I20260809 15:07:03.433774 25450 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab.
Entry for principal kudu/127.24.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab.
I20260809 15:07:03.482627 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.193:0
--local_ip_for_outbound_sockets=127.24.183.193
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:34303
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:41975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:03.814373 25470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:03.814669 25470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:03.814821 25470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:03.822384 25470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:03.822536 25470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:03.822702 25470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.193
I20260809 15:07:03.832899 25470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:41975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:34303
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.193
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:03.834264 25470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:03.835765 25470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:03.848665 25477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:03.848762 25476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:03.848791 25480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:03.850109 25470 server_base.cc:1034] running on GCE node
I20260809 15:07:03.850747 25470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:03.851658 25470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:03.852895 25470 hybrid_clock.cc:648] HybridClock initialized: now 1786288023852846 us; error 42 us; skew 500 ppm
I20260809 15:07:03.853196 25470 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:03 dist-test-slave-nj21 krb5kdc[25318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288023, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:03.856279 25470 init.cc:377] Logged in from keytab as kudu/127.24.183.193@KRBTEST.COM (short username kudu)
I20260809 15:07:03.857788 25470 webserver.cc:460] Webserver started at http://127.24.183.193:38715/ using document root <none> and password file <none>
I20260809 15:07:03.858541 25470 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:03.858678 25470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:03.858997 25470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:03.862015 25470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-0/data/instance:
uuid: "40b428c0cc0d4dc788fca1ec9678f3ad"
format_stamp: "Formatted at 2026-08-09 15:07:03 on dist-test-slave-nj21"
server_key: "726f6270ac280b1182da63ba1e3fb07c"
server_key_iv: "e9f61083897cebb99a7f6a67c6e071c7"
server_key_version: "encryptionkey@0"
I20260809 15:07:03.862776 25470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance:
uuid: "40b428c0cc0d4dc788fca1ec9678f3ad"
format_stamp: "Formatted at 2026-08-09 15:07:03 on dist-test-slave-nj21"
server_key: "726f6270ac280b1182da63ba1e3fb07c"
server_key_iv: "e9f61083897cebb99a7f6a67c6e071c7"
server_key_version: "encryptionkey@0"
I20260809 15:07:03.868098 25470 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 15:07:03.871574 25488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:03.872916 25470 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:07:03.873075 25470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-0/wal
uuid: "40b428c0cc0d4dc788fca1ec9678f3ad"
format_stamp: "Formatted at 2026-08-09 15:07:03 on dist-test-slave-nj21"
server_key: "726f6270ac280b1182da63ba1e3fb07c"
server_key_iv: "e9f61083897cebb99a7f6a67c6e071c7"
server_key_version: "encryptionkey@0"
I20260809 15:07:03.873274 25470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:03.898000 25470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:03.902189 25470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:03.902488 25470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:03.903481 25470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:07:03.904917 25470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:07:03.904999 25470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:03.905134 25470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:07:03.905190 25470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:03.937124 25470 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.193:40277
I20260809 15:07:03.937172 25644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.193:40277 every 8 connection(s)
I20260809 15:07:03.938403 25470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
I20260809 15:07:03.939450 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 25470
I20260809 15:07:03.939747 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance
I20260809 15:07:03.940653 25311 external_mini_cluster.cc:1428] Setting key 5845485a8602213ba8f0499034159a56
WARNING: no policy specified for kudu/127.24.183.194@KRBTEST.COM; defaulting to no policy
Aug 09 15:07:03 dist-test-slave-nj21 krb5kdc[25318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288023, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.194@KRBTEST.COM" created.
I20260809 15:07:03.965237 25647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:03.941652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:56075 (local address 127.24.183.254:34303)
0809 15:07:03.942124 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:03.942128 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:03.943259 (+ 1131us) server_negotiation.cc:407] Connection header received
0809 15:07:03.945806 (+ 2547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:03.945810 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:03.945883 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:03.945973 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:03.947916 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:03.948698 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:03.950272 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:03.950675 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:03.950798 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:03.959556 (+ 8758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:03.959561 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:03.959565 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:03.959600 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:03.962096 (+ 2496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:03.962963 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:03.962967 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:03.962969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:03.963080 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:03.963517 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:03.963520 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:03.963522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:03.963832 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:03.963946 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:03.964773 (+ 827us) server_negotiation.cc:299] Negotiation successful
0809 15:07:03.964949 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":268,"threads_started":1}
I20260809 15:07:03.966650 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:03.942227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:34303 (local address 127.24.183.193:56075)
0809 15:07:03.943080 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:03.943141 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:03.945534 (+ 2393us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:03.946233 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:03.946245 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:03.946971 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:03.947755 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:03.947779 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:03.948859 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:03.948865 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:03.950142 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:03.950154 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:03.950924 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:03.950932 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:03.951157 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:03.956902 (+ 5745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:03.956926 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:03.959330 (+ 2404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:03.962245 (+ 2915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:03.962256 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:03.962275 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:03.962858 (+ 583us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:03.963171 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:03.963176 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:03.963179 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:03.963426 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:03.963954 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:03.963961 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:03.964392 (+ 431us) client_negotiation.cc:769] Sending connection context
0809 15:07:03.964684 (+ 292us) client_negotiation.cc:240] Negotiation successful
0809 15:07:03.965221 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":346,"threads_started":1}
I20260809 15:07:03.968235 25645 heartbeater.cc:347] Connected to a master server at 127.24.183.254:34303
I20260809 15:07:03.968608 25645 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:03.969568 25645 heartbeater.cc:511] Master 127.24.183.254:34303 requested a full tablet report, sending...
I20260809 15:07:03.972011 25380 ts_manager.cc:194] Registered new tserver with Master: 40b428c0cc0d4dc788fca1ec9678f3ad (127.24.183.193:40277)
I20260809 15:07:03.973587 25380 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.193@KRBTEST.COM'} at 127.24.183.193:56075
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab.
Entry for principal kudu/127.24.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab.
I20260809 15:07:03.983125 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.194:0
--local_ip_for_outbound_sockets=127.24.183.194
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:34303
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:41975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:04.323246 25654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:04.323503 25654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:04.323608 25654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:04.330773 25654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:04.330871 25654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:04.330991 25654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.194
I20260809 15:07:04.340658 25654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:41975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:34303
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.194
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:04.341789 25654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:04.343043 25654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:04.356051 25661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:04.355985 25662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:04.356143 25666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:04.357007 25654 server_base.cc:1034] running on GCE node
I20260809 15:07:04.357599 25654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:04.358450 25654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:04.359644 25654 hybrid_clock.cc:648] HybridClock initialized: now 1786288024359596 us; error 42 us; skew 500 ppm
I20260809 15:07:04.359965 25654 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:04 dist-test-slave-nj21 krb5kdc[25318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288024, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:04.362946 25654 init.cc:377] Logged in from keytab as kudu/127.24.183.194@KRBTEST.COM (short username kudu)
I20260809 15:07:04.364382 25654 webserver.cc:460] Webserver started at http://127.24.183.194:37789/ using document root <none> and password file <none>
I20260809 15:07:04.365012 25654 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:04.365130 25654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:04.365424 25654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:04.368227 25654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-1/data/instance:
uuid: "b1929a035fb24880b768c96625c306a5"
format_stamp: "Formatted at 2026-08-09 15:07:04 on dist-test-slave-nj21"
server_key: "9fb0527a4ef5cf0e24bd5daff33e282a"
server_key_iv: "67af8aa372f0f3a831028cf6d119bf4a"
server_key_version: "encryptionkey@0"
I20260809 15:07:04.368937 25654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance:
uuid: "b1929a035fb24880b768c96625c306a5"
format_stamp: "Formatted at 2026-08-09 15:07:04 on dist-test-slave-nj21"
server_key: "9fb0527a4ef5cf0e24bd5daff33e282a"
server_key_iv: "67af8aa372f0f3a831028cf6d119bf4a"
server_key_version: "encryptionkey@0"
I20260809 15:07:04.373886 25654 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 15:07:04.377192 25675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:04.378540 25654 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 15:07:04.378670 25654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-1/wal
uuid: "b1929a035fb24880b768c96625c306a5"
format_stamp: "Formatted at 2026-08-09 15:07:04 on dist-test-slave-nj21"
server_key: "9fb0527a4ef5cf0e24bd5daff33e282a"
server_key_iv: "67af8aa372f0f3a831028cf6d119bf4a"
server_key_version: "encryptionkey@0"
I20260809 15:07:04.378847 25654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:04.396678 25654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:04.400657 25654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:04.400902 25654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:04.401799 25654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:07:04.403123 25654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:07:04.403198 25654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:04.403291 25654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:07:04.403337 25654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:04.434479 25654 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.194:33931
I20260809 15:07:04.434522 25838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.194:33931 every 8 connection(s)
I20260809 15:07:04.435719 25654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
I20260809 15:07:04.438519 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 25654
I20260809 15:07:04.438719 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance
I20260809 15:07:04.439277 25311 external_mini_cluster.cc:1428] Setting key b59a785064dfe5240e977785d9140200
Aug 09 15:07:04 dist-test-slave-nj21 krb5kdc[25318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288024, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.183.195@KRBTEST.COM; defaulting to no policy
I20260809 15:07:04.455488 25647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:04.438256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:47235 (local address 127.24.183.254:34303)
0809 15:07:04.438425 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:04.438429 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:04.439890 (+ 1461us) server_negotiation.cc:407] Connection header received
0809 15:07:04.442023 (+ 2133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:04.442028 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:04.442106 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:04.442192 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:04.443697 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:04.444559 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:04.446059 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:04.446510 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:04.446662 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:04.449992 (+ 3330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:04.449998 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:04.450001 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:04.450035 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:04.452608 (+ 2573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:04.453292 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:04.453298 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:04.453301 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:04.453436 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:04.453836 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:04.453841 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:04.453843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:04.454110 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:04.454268 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:04.454898 (+ 630us) server_negotiation.cc:299] Negotiation successful
0809 15:07:04.455134 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 15:07:04.456537 25843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:04.438899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:34303 (local address 127.24.183.194:47235)
0809 15:07:04.439716 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:04.439764 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:04.441759 (+ 1995us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:04.442436 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:04.442444 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:04.442966 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:04.443502 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:04.443519 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:04.444709 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:04.444713 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:04.445918 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:04.445931 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:04.446744 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:04.446751 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:04.446982 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:04.447617 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:04.447644 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:04.449769 (+ 2125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:04.452743 (+ 2974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:04.452752 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:04.452767 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:04.453177 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:04.453539 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:04.453544 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:04.453548 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:04.453730 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:04.454234 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:04.454241 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:04.454556 (+ 315us) client_negotiation.cc:769] Sending connection context
0809 15:07:04.454792 (+ 236us) client_negotiation.cc:240] Negotiation successful
0809 15:07:04.455301 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":314,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.195@KRBTEST.COM" created.
I20260809 15:07:04.458035 25839 heartbeater.cc:347] Connected to a master server at 127.24.183.254:34303
I20260809 15:07:04.458330 25839 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:04.459076 25839 heartbeater.cc:511] Master 127.24.183.254:34303 requested a full tablet report, sending...
I20260809 15:07:04.460525 25380 ts_manager.cc:194] Registered new tserver with Master: b1929a035fb24880b768c96625c306a5 (127.24.183.194:33931)
I20260809 15:07:04.461326 25380 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.194@KRBTEST.COM'} at 127.24.183.194:47235
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab.
Entry for principal kudu/127.24.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab.
I20260809 15:07:04.478328 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.195:0
--local_ip_for_outbound_sockets=127.24.183.195
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:34303
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:41975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:04.813937 25849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:04.814204 25849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:04.814324 25849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:04.821782 25849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:04.821909 25849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:04.822043 25849 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.195
I20260809 15:07:04.832116 25849 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:41975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:34303
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.195
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:04.833366 25849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:04.834692 25849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:04.847474 25854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:04.847440 25858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:04.847555 25860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:04.848485 25849 server_base.cc:1034] running on GCE node
I20260809 15:07:04.849085 25849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:04.850003 25849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:04.851202 25849 hybrid_clock.cc:648] HybridClock initialized: now 1786288024851145 us; error 47 us; skew 500 ppm
I20260809 15:07:04.851495 25849 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:04 dist-test-slave-nj21 krb5kdc[25318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288024, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:04.854542 25849 init.cc:377] Logged in from keytab as kudu/127.24.183.195@KRBTEST.COM (short username kudu)
I20260809 15:07:04.855952 25849 webserver.cc:460] Webserver started at http://127.24.183.195:40539/ using document root <none> and password file <none>
I20260809 15:07:04.856564 25849 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:04.856662 25849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:04.856936 25849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:04.859803 25849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-2/data/instance:
uuid: "d22b41ee73f14a179da91bd736dd91be"
format_stamp: "Formatted at 2026-08-09 15:07:04 on dist-test-slave-nj21"
server_key: "c697a7b4fb5ec8299ef32a24d2be1904"
server_key_iv: "121f0682318d2a1d7fde7f477daa806f"
server_key_version: "encryptionkey@0"
I20260809 15:07:04.860579 25849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance:
uuid: "d22b41ee73f14a179da91bd736dd91be"
format_stamp: "Formatted at 2026-08-09 15:07:04 on dist-test-slave-nj21"
server_key: "c697a7b4fb5ec8299ef32a24d2be1904"
server_key_iv: "121f0682318d2a1d7fde7f477daa806f"
server_key_version: "encryptionkey@0"
I20260809 15:07:04.865571 25849 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 15:07:04.868980 25872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:04.870277 25849 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:07:04.870406 25849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-2/wal
uuid: "d22b41ee73f14a179da91bd736dd91be"
format_stamp: "Formatted at 2026-08-09 15:07:04 on dist-test-slave-nj21"
server_key: "c697a7b4fb5ec8299ef32a24d2be1904"
server_key_iv: "121f0682318d2a1d7fde7f477daa806f"
server_key_version: "encryptionkey@0"
I20260809 15:07:04.870564 25849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:04.889048 25849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:04.893199 25849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:04.893447 25849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:04.894397 25849 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:07:04.895744 25849 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:07:04.895833 25849 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:04.895962 25849 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:07:04.896013 25849 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:04.928395 25849 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.195:45011
I20260809 15:07:04.928429 26040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.195:45011 every 8 connection(s)
I20260809 15:07:04.929704 25849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
I20260809 15:07:04.933516 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 25849
I20260809 15:07:04.933738 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance
I20260809 15:07:04.934306 25311 external_mini_cluster.cc:1428] Setting key ecbd8d9ed174e203b4d9000ef894332e
Aug 09 15:07:04 dist-test-slave-nj21 krb5kdc[25318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288024, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:04.950701 25647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:04.932342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:52535 (local address 127.24.183.254:34303)
0809 15:07:04.932512 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:04.932517 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:04.933845 (+ 1328us) server_negotiation.cc:407] Connection header received
0809 15:07:04.936182 (+ 2337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:04.936186 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:04.936281 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:04.936388 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:04.938472 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:04.939606 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:04.941226 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:04.941836 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:04.941962 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:04.945111 (+ 3149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:04.945116 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:04.945120 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:04.945154 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:04.947661 (+ 2507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:04.948297 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:04.948301 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:04.948303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:04.948433 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:04.949012 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:04.949017 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:04.949020 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:04.949335 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:04.949463 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:04.950223 (+ 760us) server_negotiation.cc:299] Negotiation successful
0809 15:07:04.950418 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 15:07:04.951648 26047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:04.932781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:34303 (local address 127.24.183.195:52535)
0809 15:07:04.933630 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:04.933683 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:04.935923 (+ 2240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:04.936642 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:04.936653 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:04.937348 (+ 695us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:04.938264 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:04.938293 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:04.939800 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:04.939805 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:04.941082 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:04.941093 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:04.942046 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:04.942051 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:04.942248 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:04.942837 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:04.942859 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:04.944936 (+ 2077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:04.947803 (+ 2867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:04.947809 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:04.947818 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:04.948193 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:04.948564 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:04.948569 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:04.948571 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:04.948885 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:04.949468 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:04.949475 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:04.949831 (+ 356us) client_negotiation.cc:769] Sending connection context
0809 15:07:04.950108 (+ 277us) client_negotiation.cc:240] Negotiation successful
0809 15:07:04.950543 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":316,"threads_started":1}
I20260809 15:07:04.952859 26043 heartbeater.cc:347] Connected to a master server at 127.24.183.254:34303
I20260809 15:07:04.953127 26043 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:04.953805 26043 heartbeater.cc:511] Master 127.24.183.254:34303 requested a full tablet report, sending...
I20260809 15:07:04.955076 25380 ts_manager.cc:194] Registered new tserver with Master: d22b41ee73f14a179da91bd736dd91be (127.24.183.195:45011)
I20260809 15:07:04.955874 25380 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.195@KRBTEST.COM'} at 127.24.183.195:52535
I20260809 15:07:04.962704 25311 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:07:04.976760 25645 heartbeater.cc:503] Master 127.24.183.254:34303 was elected leader, sending a full tablet report...
Aug 09 15:07:04 dist-test-slave-nj21 krb5kdc[25318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288024, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 15:07:04 dist-test-slave-nj21 krb5kdc[25318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288024, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.183.193@KRBTEST.COM
I20260809 15:07:05.007198 26056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:04.992670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49974 (local address 127.24.183.193:40277)
0809 15:07:04.993345 (+ 675us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:04.993352 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:04.993380 (+ 28us) server_negotiation.cc:407] Connection header received
0809 15:07:04.993479 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:04.993486 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:04.993693 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:04.993852 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:04.995348 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:04.996433 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:04.997673 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:04.998229 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:04.998326 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:05.002080 (+ 3754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:05.002091 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:05.002106 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:05.002142 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:05.004813 (+ 2671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:05.005428 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:05.005435 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:05.005437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:05.005569 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:05.005915 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:05.005918 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:05.005920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:05.006244 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:05.006469 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:05.006694 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 15:07:05.006905 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":347,"threads_started":1}
W20260809 15:07:05.008488 25524 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49974
Aug 09 15:07:05 dist-test-slave-nj21 krb5kdc[25318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 15:07:05 dist-test-slave-nj21 krb5kdc[25318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.193@KRBTEST.COM
I20260809 15:07:05.052806 26056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:05.039577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49980 (local address 127.24.183.193:40277)
0809 15:07:05.039813 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:05.039820 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:05.040084 (+ 264us) server_negotiation.cc:407] Connection header received
0809 15:07:05.040233 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:05.040237 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:05.040346 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:05.040433 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:05.041794 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:05.042503 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:05.043667 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:05.044130 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:05.044234 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:05.047866 (+ 3632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:05.047870 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:05.047873 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:05.047906 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:05.050432 (+ 2526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:05.051023 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:05.051027 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:05.051030 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:05.051155 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:05.051490 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:05.051494 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:05.051498 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:05.051772 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:05.051998 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:05.052291 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 15:07:05.052505 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 15:07:05.054171 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 25470
I20260809 15:07:05.065518 26056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:05.055050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49992 (local address 127.24.183.193:40277)
0809 15:07:05.055188 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:05.055192 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:05.055519 (+ 327us) server_negotiation.cc:407] Connection header received
0809 15:07:05.055622 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:05.055625 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:05.055695 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:05.055772 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:05.056847 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:05.057490 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:05.058657 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:05.059127 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:05.059237 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:05.061211 (+ 1974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:05.061215 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:05.061218 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:05.061246 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:05.063556 (+ 2310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:05.064137 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:05.064140 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:05.064143 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:05.064263 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:05.064511 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:05.064514 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:05.064516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:05.064738 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:05.064930 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:05.065083 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 15:07:05.065250 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 15:07:05.066378 25637 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:07:05.150264 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 25470
I20260809 15:07:05.172533 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 25654
Aug 09 15:07:05 dist-test-slave-nj21 krb5kdc[25318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.194@KRBTEST.COM
I20260809 15:07:05.189523 26067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:05.173579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38088 (local address 127.24.183.194:33931)
0809 15:07:05.174206 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:05.174213 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:05.174242 (+ 29us) server_negotiation.cc:407] Connection header received
0809 15:07:05.174317 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:05.174325 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:05.174533 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:05.174675 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:05.175989 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:05.177128 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:05.178374 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:05.178916 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:05.179014 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:05.183096 (+ 4082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:05.183108 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:05.183123 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:05.183167 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:05.186209 (+ 3042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:05.186892 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:05.186902 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:05.186905 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:05.187073 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:05.187582 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:05.187587 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:05.187591 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:05.188123 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:05.188452 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:05.188783 (+ 331us) server_negotiation.cc:299] Negotiation successful
0809 15:07:05.189082 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":321,"threads_started":1}
I20260809 15:07:05.190936 25833 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:07:05.273200 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 25654
I20260809 15:07:05.295418 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 25849
Aug 09 15:07:05 dist-test-slave-nj21 krb5kdc[25318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.195@KRBTEST.COM
I20260809 15:07:05.310971 26069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:05.296525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51300 (local address 127.24.183.195:45011)
0809 15:07:05.297222 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:05.297229 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:05.297261 (+ 32us) server_negotiation.cc:407] Connection header received
0809 15:07:05.297329 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:05.297336 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:05.297554 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:05.297687 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:05.298876 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:05.300034 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:05.301252 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:05.301815 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:05.301905 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:05.305891 (+ 3986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:05.305902 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:05.305925 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:05.305958 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:05.308618 (+ 2660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:05.309187 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:05.309193 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:05.309196 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:05.309317 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:05.309604 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:05.309606 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:05.309608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:05.309939 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:05.310172 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:05.310404 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 15:07:05.310598 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":376,"threads_started":1}
I20260809 15:07:05.312214 26031 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:07:05.394060 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 25849
I20260809 15:07:05.415406 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 25334
I20260809 15:07:05.416644 25431 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:07:05.521306 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 25334
2026-08-09T15:07:05Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4174 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:07:05 dist-test-slave-nj21 krb5kdc[26076](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:07:05 dist-test-slave-nj21 krb5kdc[26076](info): set up 2 sockets
Aug 09 15:07:05 dist-test-slave-nj21 krb5kdc[26076](info): commencing operation
krb5kdc: starting...
W20260809 15:07:06.707013 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.000s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:07:06 dist-test-slave-nj21 krb5kdc[26076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:07:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:07:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
I20260809 15:07:06.883327 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:45917
--webserver_interface=127.24.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:45917
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:07.206332 26097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:07.206617 26097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:07.206756 26097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:07.213853 26097 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:07:07.213959 26097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:07.214040 26097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:07.214114 26097 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:07:07.214172 26097 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:07:07.223881 26097 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:45917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:45917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:07.225173 26097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:07.226437 26097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:07.237555 26110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:07.237623 26103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:07.237522 26106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:07.238358 26097 server_base.cc:1034] running on GCE node
I20260809 15:07:07.239053 26097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:07.240377 26097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:07.241576 26097 hybrid_clock.cc:648] HybridClock initialized: now 1786288027241528 us; error 32 us; skew 500 ppm
I20260809 15:07:07.241921 26097 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:07 dist-test-slave-nj21 krb5kdc[26076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288027, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:07.249389 26097 init.cc:377] Logged in from keytab as kudu/127.24.183.254@KRBTEST.COM (short username kudu)
I20260809 15:07:07.250784 26097 webserver.cc:460] Webserver started at http://127.24.183.254:38431/ using document root <none> and password file <none>
I20260809 15:07:07.251446 26097 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:07.251569 26097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:07.251915 26097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:07.254849 26097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "24e108b559f74e6b9a3813a104fd2bc9"
format_stamp: "Formatted at 2026-08-09 15:07:07 on dist-test-slave-nj21"
server_key: "719e2060c83805f6a6ab03a0f37c3f05"
server_key_iv: "c10a81ba0e8b65417022863e0f331292"
server_key_version: "encryptionkey@0"
I20260809 15:07:07.255580 26097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "24e108b559f74e6b9a3813a104fd2bc9"
format_stamp: "Formatted at 2026-08-09 15:07:07 on dist-test-slave-nj21"
server_key: "719e2060c83805f6a6ab03a0f37c3f05"
server_key_iv: "c10a81ba0e8b65417022863e0f331292"
server_key_version: "encryptionkey@0"
I20260809 15:07:07.260589 26097 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260809 15:07:07.263864 26121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:07.265094 26097 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:07:07.265254 26097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "24e108b559f74e6b9a3813a104fd2bc9"
format_stamp: "Formatted at 2026-08-09 15:07:07 on dist-test-slave-nj21"
server_key: "719e2060c83805f6a6ab03a0f37c3f05"
server_key_iv: "c10a81ba0e8b65417022863e0f331292"
server_key_version: "encryptionkey@0"
I20260809 15:07:07.265439 26097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:07.299609 26097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:07.303481 26097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:07.303779 26097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:07.321521 26097 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:45917
I20260809 15:07:07.321543 26208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:45917 every 8 connection(s)
I20260809 15:07:07.322860 26097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:07:07.326484 26209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:07.329133 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 26097
I20260809 15:07:07.329376 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:07:07.330173 25311 external_mini_cluster.cc:1428] Setting key 5bb40a4ae2122fdc8c81298ad956152f
I20260809 15:07:07.338671 26209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24e108b559f74e6b9a3813a104fd2bc9: Bootstrap starting.
I20260809 15:07:07.342320 26209 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24e108b559f74e6b9a3813a104fd2bc9: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:07.343564 26209 log.cc:824] T 00000000000000000000000000000000 P 24e108b559f74e6b9a3813a104fd2bc9: Log is configured to *not* fsync() on all Append() calls
Aug 09 15:07:07 dist-test-slave-nj21 krb5kdc[26076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:07.346735 26209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24e108b559f74e6b9a3813a104fd2bc9: No bootstrap required, opened a new log
I20260809 15:07:07.354341 26213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:07.332249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35904 (local address 127.24.183.254:45917)
0809 15:07:07.333040 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:07.333051 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:07.333093 (+ 42us) server_negotiation.cc:407] Connection header received
0809 15:07:07.334760 (+ 1667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:07.334782 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:07.335158 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:07.335469 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:07.337285 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:07.338558 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:07.340245 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:07.340918 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:07.341048 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:07.345811 (+ 4763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:07.345823 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:07.345842 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:07.345886 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:07.349878 (+ 3992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:07.350547 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:07.350556 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:07.350564 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:07.350773 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:07.351083 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:07.351088 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:07.351092 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:07.351837 (+ 745us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:07.352151 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:07.352516 (+ 365us) server_negotiation.cc:299] Negotiation successful
0809 15:07:07.352943 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":371,"threads_started":1}
I20260809 15:07:07.357014 26209 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24e108b559f74e6b9a3813a104fd2bc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24e108b559f74e6b9a3813a104fd2bc9" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45917 } }
I20260809 15:07:07.357455 26209 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24e108b559f74e6b9a3813a104fd2bc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:07.357537 26209 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24e108b559f74e6b9a3813a104fd2bc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24e108b559f74e6b9a3813a104fd2bc9, State: Initialized, Role: FOLLOWER
I20260809 15:07:07.358026 26209 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24e108b559f74e6b9a3813a104fd2bc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24e108b559f74e6b9a3813a104fd2bc9" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45917 } }
I20260809 15:07:07.358229 26209 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24e108b559f74e6b9a3813a104fd2bc9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:07:07.358330 26209 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24e108b559f74e6b9a3813a104fd2bc9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:07:07.358444 26209 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24e108b559f74e6b9a3813a104fd2bc9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:07.360719 26209 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24e108b559f74e6b9a3813a104fd2bc9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24e108b559f74e6b9a3813a104fd2bc9" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45917 } }
I20260809 15:07:07.361147 26209 leader_election.cc:302] T 00000000000000000000000000000000 P 24e108b559f74e6b9a3813a104fd2bc9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24e108b559f74e6b9a3813a104fd2bc9; no voters:
I20260809 15:07:07.361658 26209 leader_election.cc:288] T 00000000000000000000000000000000 P 24e108b559f74e6b9a3813a104fd2bc9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:07:07.361775 26215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24e108b559f74e6b9a3813a104fd2bc9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:07:07.362396 26215 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24e108b559f74e6b9a3813a104fd2bc9 [term 1 LEADER]: Becoming Leader. State: Replica: 24e108b559f74e6b9a3813a104fd2bc9, State: Running, Role: LEADER
I20260809 15:07:07.362896 26215 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24e108b559f74e6b9a3813a104fd2bc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24e108b559f74e6b9a3813a104fd2bc9" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45917 } }
I20260809 15:07:07.363615 26209 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24e108b559f74e6b9a3813a104fd2bc9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:07:07.365477 26218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24e108b559f74e6b9a3813a104fd2bc9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24e108b559f74e6b9a3813a104fd2bc9. Latest consensus state: current_term: 1 leader_uuid: "24e108b559f74e6b9a3813a104fd2bc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24e108b559f74e6b9a3813a104fd2bc9" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45917 } } }
I20260809 15:07:07.365969 26218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24e108b559f74e6b9a3813a104fd2bc9 [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:07.365921 26216 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24e108b559f74e6b9a3813a104fd2bc9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24e108b559f74e6b9a3813a104fd2bc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24e108b559f74e6b9a3813a104fd2bc9" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45917 } } }
I20260809 15:07:07.366261 26216 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24e108b559f74e6b9a3813a104fd2bc9 [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:07.368983 26226 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:07:07.381505 26226 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:07:07.390292 26226 catalog_manager.cc:1347] Generated new cluster ID: e533a5a014b040f5b980c7a03dfb442a
I20260809 15:07:07.390419 26226 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:07:07.405550 26226 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:07:07.406912 26226 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:07:07.416565 26226 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24e108b559f74e6b9a3813a104fd2bc9: Generated new TSK 0
I20260809 15:07:07.417564 26226 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab.
Entry for principal kudu/127.24.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab.
I20260809 15:07:07.468334 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.193:0
--local_ip_for_outbound_sockets=127.24.183.193
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:45917
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:07.789558 26245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:07.789840 26245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:07.789988 26245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:07.797150 26245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:07.797261 26245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:07.797422 26245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.193
I20260809 15:07:07.806694 26245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:45917
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.193
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:07.807953 26245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:07.809242 26245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:07.822111 26257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:07.822044 26259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:07.822078 26262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:07.822927 26245 server_base.cc:1034] running on GCE node
I20260809 15:07:07.823546 26245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:07.824740 26245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:07.825963 26245 hybrid_clock.cc:648] HybridClock initialized: now 1786288027825904 us; error 42 us; skew 500 ppm
I20260809 15:07:07.826288 26245 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:07 dist-test-slave-nj21 krb5kdc[26076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288027, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:07.829304 26245 init.cc:377] Logged in from keytab as kudu/127.24.183.193@KRBTEST.COM (short username kudu)
I20260809 15:07:07.830677 26245 webserver.cc:460] Webserver started at http://127.24.183.193:33355/ using document root <none> and password file <none>
I20260809 15:07:07.831326 26245 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:07.831445 26245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:07.831777 26245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:07.834545 26245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-0/data/instance:
uuid: "92b7225b8e97491bab64f4f4fe62b0d7"
format_stamp: "Formatted at 2026-08-09 15:07:07 on dist-test-slave-nj21"
server_key: "288dc44322cbb1d0d9de93fa9085ce10"
server_key_iv: "5f2494de8b9bcd0424ebfac93c8154ae"
server_key_version: "encryptionkey@0"
I20260809 15:07:07.835310 26245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance:
uuid: "92b7225b8e97491bab64f4f4fe62b0d7"
format_stamp: "Formatted at 2026-08-09 15:07:07 on dist-test-slave-nj21"
server_key: "288dc44322cbb1d0d9de93fa9085ce10"
server_key_iv: "5f2494de8b9bcd0424ebfac93c8154ae"
server_key_version: "encryptionkey@0"
I20260809 15:07:07.840130 26245 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260809 15:07:07.843459 26270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:07.844667 26245 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:07:07.844823 26245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-0/wal
uuid: "92b7225b8e97491bab64f4f4fe62b0d7"
format_stamp: "Formatted at 2026-08-09 15:07:07 on dist-test-slave-nj21"
server_key: "288dc44322cbb1d0d9de93fa9085ce10"
server_key_iv: "5f2494de8b9bcd0424ebfac93c8154ae"
server_key_version: "encryptionkey@0"
I20260809 15:07:07.844995 26245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:07.855389 26245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:07.859052 26245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:07.859302 26245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:07.860209 26245 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:07:07.861443 26245 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:07:07.861521 26245 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:07.861639 26245 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:07:07.861691 26245 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:07.892294 26245 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.193:44039
I20260809 15:07:07.892346 26434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.193:44039 every 8 connection(s)
I20260809 15:07:07.893541 26245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
I20260809 15:07:07.903900 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 26245
I20260809 15:07:07.904104 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance
I20260809 15:07:07.904706 25311 external_mini_cluster.cc:1428] Setting key 02a7ee6908e19bfaf3f4b9d0baafe43a
Aug 09 15:07:07 dist-test-slave-nj21 krb5kdc[26076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288027, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:07.913116 26439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:07.896008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:33967 (local address 127.24.183.254:45917)
0809 15:07:07.896426 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:07.896430 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:07.897385 (+ 955us) server_negotiation.cc:407] Connection header received
0809 15:07:07.899206 (+ 1821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:07.899212 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:07.899299 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:07.899396 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:07.900908 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:07.901653 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:07.903138 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:07.903545 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:07.903655 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:07.907155 (+ 3500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:07.907161 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:07.907166 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:07.907207 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:07.910268 (+ 3061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:07.910938 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:07.910944 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:07.910947 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:07.911088 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:07.911466 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:07.911470 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:07.911473 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:07.911848 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:07.911984 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:07.912608 (+ 624us) server_negotiation.cc:299] Negotiation successful
0809 15:07:07.912843 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":214,"threads_started":1}
I20260809 15:07:07.914170 26440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:07.896545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45917 (local address 127.24.183.193:33967)
0809 15:07:07.897219 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:07.897263 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:07.898950 (+ 1687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:07.899638 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:07.899645 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:07.900184 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:07.900741 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:07.900759 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:07.901795 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:07.901799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:07.902996 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:07.903013 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:07.903784 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:07.903793 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:07.904020 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:07.904653 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:07.904675 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:07.906944 (+ 2269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:07.910394 (+ 3450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:07.910400 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:07.910412 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:07.910822 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:07.911174 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:07.911177 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:07.911180 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:07.911365 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:07.911950 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:07.911955 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:07.912254 (+ 299us) client_negotiation.cc:769] Sending connection context
0809 15:07:07.912481 (+ 227us) client_negotiation.cc:240] Negotiation successful
0809 15:07:07.912855 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":258,"threads_started":1}
I20260809 15:07:07.915642 26437 heartbeater.cc:347] Connected to a master server at 127.24.183.254:45917
I20260809 15:07:07.915975 26437 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:07.916814 26437 heartbeater.cc:511] Master 127.24.183.254:45917 requested a full tablet report, sending...
I20260809 15:07:07.919145 26146 ts_manager.cc:194] Registered new tserver with Master: 92b7225b8e97491bab64f4f4fe62b0d7 (127.24.183.193:44039)
I20260809 15:07:07.921169 26146 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.193@KRBTEST.COM'} at 127.24.183.193:33967
WARNING: no policy specified for kudu/127.24.183.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab.
Entry for principal kudu/127.24.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab.
I20260809 15:07:07.946973 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.194:0
--local_ip_for_outbound_sockets=127.24.183.194
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:45917
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:08.274855 26443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:08.275115 26443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:08.275226 26443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:08.282246 26443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:08.282348 26443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:08.282467 26443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.194
I20260809 15:07:08.291764 26443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:45917
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.194
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:08.292971 26443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:08.294252 26443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:08.307447 26454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:08.307406 26450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:08.307473 26451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:08.308475 26443 server_base.cc:1034] running on GCE node
I20260809 15:07:08.309085 26443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:08.309942 26443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:08.311148 26443 hybrid_clock.cc:648] HybridClock initialized: now 1786288028311107 us; error 29 us; skew 500 ppm
I20260809 15:07:08.311431 26443 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:08 dist-test-slave-nj21 krb5kdc[26076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288028, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:08.314453 26443 init.cc:377] Logged in from keytab as kudu/127.24.183.194@KRBTEST.COM (short username kudu)
I20260809 15:07:08.315948 26443 webserver.cc:460] Webserver started at http://127.24.183.194:35175/ using document root <none> and password file <none>
I20260809 15:07:08.316560 26443 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:08.316663 26443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:08.316927 26443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:08.319734 26443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-1/data/instance:
uuid: "7cd9ffa189f54c65b7fbbcfdf2437123"
format_stamp: "Formatted at 2026-08-09 15:07:08 on dist-test-slave-nj21"
server_key: "96e20ec2b2607cbf1ad74ae0d26bd95c"
server_key_iv: "5c063ff1444085412d7f33074f9d56ba"
server_key_version: "encryptionkey@0"
I20260809 15:07:08.320475 26443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance:
uuid: "7cd9ffa189f54c65b7fbbcfdf2437123"
format_stamp: "Formatted at 2026-08-09 15:07:08 on dist-test-slave-nj21"
server_key: "96e20ec2b2607cbf1ad74ae0d26bd95c"
server_key_iv: "5c063ff1444085412d7f33074f9d56ba"
server_key_version: "encryptionkey@0"
I20260809 15:07:08.325942 26443 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 15:07:08.329332 26465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:08.330653 26443 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 15:07:08.330821 26443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-1/wal
uuid: "7cd9ffa189f54c65b7fbbcfdf2437123"
format_stamp: "Formatted at 2026-08-09 15:07:08 on dist-test-slave-nj21"
server_key: "96e20ec2b2607cbf1ad74ae0d26bd95c"
server_key_iv: "5c063ff1444085412d7f33074f9d56ba"
server_key_version: "encryptionkey@0"
I20260809 15:07:08.331000 26443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:08.357658 26443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:08.361637 26443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:08.361929 26443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:08.362898 26443 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:07:08.364250 26443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:07:08.364329 26443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:08.364451 26443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:07:08.364513 26443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:08.396461 26443 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.194:33381
I20260809 15:07:08.396517 26625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.194:33381 every 8 connection(s)
I20260809 15:07:08.397787 26443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
I20260809 15:07:08.402354 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 26443
I20260809 15:07:08.402542 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance
I20260809 15:07:08.402988 25311 external_mini_cluster.cc:1428] Setting key bcc824e8984a569530fd60caf841f376
Aug 09 15:07:08 dist-test-slave-nj21 krb5kdc[26076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288028, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:08.417557 26439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:08.400387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:49355 (local address 127.24.183.254:45917)
0809 15:07:08.400559 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:08.400563 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:08.401710 (+ 1147us) server_negotiation.cc:407] Connection header received
0809 15:07:08.403764 (+ 2054us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:08.403770 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:08.403857 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:08.403947 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:08.405441 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:08.406180 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:08.407721 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:08.408161 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:08.408274 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:08.411871 (+ 3597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:08.411876 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:08.411879 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:08.411912 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:08.414351 (+ 2439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:08.415218 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:08.415222 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:08.415225 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:08.415355 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:08.415831 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:08.415834 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:08.415836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:08.416105 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:08.416219 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:08.417061 (+ 842us) server_negotiation.cc:299] Negotiation successful
0809 15:07:08.417241 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 15:07:08.418937 26630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:08.400800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45917 (local address 127.24.183.194:49355)
0809 15:07:08.401536 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:08.401575 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:08.403453 (+ 1878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:08.404189 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:08.404197 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:08.404708 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:08.405278 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:08.405296 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:08.406332 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:08.406336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:08.407555 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:08.407567 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:08.408393 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:08.408400 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:08.408618 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:08.409365 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:08.409395 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:08.411624 (+ 2229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:08.414488 (+ 2864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:08.414496 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:08.414510 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:08.415108 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:08.415454 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:08.415459 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:08.415462 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:08.415720 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:08.416240 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:08.416248 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:08.416663 (+ 415us) client_negotiation.cc:769] Sending connection context
0809 15:07:08.416984 (+ 321us) client_negotiation.cc:240] Negotiation successful
0809 15:07:08.417507 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":277,"threads_started":1}
I20260809 15:07:08.420351 26627 heartbeater.cc:347] Connected to a master server at 127.24.183.254:45917
I20260809 15:07:08.420701 26627 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.24.183.195@KRBTEST.COM; defaulting to no policy
I20260809 15:07:08.421567 26627 heartbeater.cc:511] Master 127.24.183.254:45917 requested a full tablet report, sending...
I20260809 15:07:08.423017 26146 ts_manager.cc:194] Registered new tserver with Master: 7cd9ffa189f54c65b7fbbcfdf2437123 (127.24.183.194:33381)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.195@KRBTEST.COM" created.
I20260809 15:07:08.423964 26146 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.194@KRBTEST.COM'} at 127.24.183.194:49355
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab.
Entry for principal kudu/127.24.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab.
I20260809 15:07:08.446583 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.195:0
--local_ip_for_outbound_sockets=127.24.183.195
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:45917
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:08.775286 26635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:08.775563 26635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:08.775730 26635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:08.782775 26635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:08.782892 26635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:08.783054 26635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.195
I20260809 15:07:08.792936 26635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:45917
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.195
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:08.794255 26635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:08.795578 26635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:08.808822 26645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:08.809118 26641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:08.809247 26643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:08.810065 26635 server_base.cc:1034] running on GCE node
I20260809 15:07:08.810689 26635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:08.811619 26635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:08.812855 26635 hybrid_clock.cc:648] HybridClock initialized: now 1786288028812797 us; error 47 us; skew 500 ppm
I20260809 15:07:08.813151 26635 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:08 dist-test-slave-nj21 krb5kdc[26076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288028, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:08.816285 26635 init.cc:377] Logged in from keytab as kudu/127.24.183.195@KRBTEST.COM (short username kudu)
I20260809 15:07:08.817710 26635 webserver.cc:460] Webserver started at http://127.24.183.195:38789/ using document root <none> and password file <none>
I20260809 15:07:08.818357 26635 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:08.818478 26635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:08.818783 26635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:08.822033 26635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-2/data/instance:
uuid: "1dd2a2b141f84eb0acf4e216d5348c2d"
format_stamp: "Formatted at 2026-08-09 15:07:08 on dist-test-slave-nj21"
server_key: "7605878fb13dc1d7eeaaa5958f096f79"
server_key_iv: "6ca16177a57749841e1692628580bc71"
server_key_version: "encryptionkey@0"
I20260809 15:07:08.822923 26635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance:
uuid: "1dd2a2b141f84eb0acf4e216d5348c2d"
format_stamp: "Formatted at 2026-08-09 15:07:08 on dist-test-slave-nj21"
server_key: "7605878fb13dc1d7eeaaa5958f096f79"
server_key_iv: "6ca16177a57749841e1692628580bc71"
server_key_version: "encryptionkey@0"
I20260809 15:07:08.828508 26635 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 15:07:08.832238 26659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:08.833568 26635 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:07:08.833736 26635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-2/wal
uuid: "1dd2a2b141f84eb0acf4e216d5348c2d"
format_stamp: "Formatted at 2026-08-09 15:07:08 on dist-test-slave-nj21"
server_key: "7605878fb13dc1d7eeaaa5958f096f79"
server_key_iv: "6ca16177a57749841e1692628580bc71"
server_key_version: "encryptionkey@0"
I20260809 15:07:08.833967 26635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:08.855451 26635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:08.859592 26635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:08.859910 26635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:08.860867 26635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:07:08.862192 26635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:07:08.862279 26635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:08.862397 26635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:07:08.862450 26635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:08.898125 26635 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.195:38647
I20260809 15:07:08.898181 26814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.195:38647 every 8 connection(s)
I20260809 15:07:08.899322 26635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
I20260809 15:07:08.902559 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 26635
I20260809 15:07:08.902758 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance
I20260809 15:07:08.903184 25311 external_mini_cluster.cc:1428] Setting key 5c2fada59b17ebfdc4808fbfa5234553
Aug 09 15:07:08 dist-test-slave-nj21 krb5kdc[26076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288028, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:08.918843 26439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:08.901753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:51109 (local address 127.24.183.254:45917)
0809 15:07:08.901982 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:08.901989 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:08.903231 (+ 1242us) server_negotiation.cc:407] Connection header received
0809 15:07:08.905418 (+ 2187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:08.905422 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:08.905488 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:08.905568 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:08.907262 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:08.908101 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:08.909635 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:08.910073 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:08.910172 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:08.913354 (+ 3182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:08.913359 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:08.913362 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:08.913396 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:08.915937 (+ 2541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:08.916521 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:08.916525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:08.916527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:08.916679 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:08.917130 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:08.917135 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:08.917138 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:08.917464 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:08.917602 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:08.918268 (+ 666us) server_negotiation.cc:299] Negotiation successful
0809 15:07:08.918505 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 15:07:08.919831 26818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:08.902212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45917 (local address 127.24.183.195:51109)
0809 15:07:08.903052 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:08.903095 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:08.905172 (+ 2077us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:08.905857 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:08.905868 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:08.906538 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:08.907095 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:08.907112 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:08.908229 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:08.908233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:08.909484 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:08.909496 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:08.910251 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:08.910256 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:08.910446 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:08.911019 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:08.911041 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:08.913151 (+ 2110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:08.916033 (+ 2882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:08.916039 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:08.916049 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:08.916438 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:08.916772 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:08.916775 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:08.916777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:08.917006 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:08.917592 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:08.917599 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:08.917919 (+ 320us) client_negotiation.cc:769] Sending connection context
0809 15:07:08.918160 (+ 241us) client_negotiation.cc:240] Negotiation successful
0809 15:07:08.918643 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":336,"threads_started":1}
I20260809 15:07:08.921200 26816 heartbeater.cc:347] Connected to a master server at 127.24.183.254:45917
I20260809 15:07:08.921479 26816 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:08.922159 26816 heartbeater.cc:511] Master 127.24.183.254:45917 requested a full tablet report, sending...
I20260809 15:07:08.923579 26146 ts_manager.cc:194] Registered new tserver with Master: 1dd2a2b141f84eb0acf4e216d5348c2d (127.24.183.195:38647)
I20260809 15:07:08.924503 26146 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.195@KRBTEST.COM'} at 127.24.183.195:51109
I20260809 15:07:08.924688 26437 heartbeater.cc:503] Master 127.24.183.254:45917 was elected leader, sending a full tablet report...
I20260809 15:07:08.930590 25311 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:07:08.950286 26439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:08.937195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35916 (local address 127.24.183.254:45917)
0809 15:07:08.937386 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:08.937391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:08.937958 (+ 567us) server_negotiation.cc:407] Connection header received
0809 15:07:08.938099 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:08.938102 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:08.938180 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:08.938254 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:08.940042 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:08.940755 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:08.942054 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:08.942447 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:08.942530 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:08.944951 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:08.944956 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:08.944960 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:08.944990 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:08.947897 (+ 2907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:08.948543 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:08.948547 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:08.948550 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:08.948680 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:08.948971 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:08.948974 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:08.948976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:08.949348 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:08.949539 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:08.949820 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 15:07:08.949982 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 15:07:08.956650 26146 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35916:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:07:08.963858 26146 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:07:08.987835 26841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:08.977984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40066 (local address 127.24.183.193:44039)
0809 15:07:08.979318 (+ 1334us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:08.979328 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:08.979843 (+ 515us) server_negotiation.cc:407] Connection header received
0809 15:07:08.980028 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:08.980038 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:08.980306 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:08.980500 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:07:08.981747 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:08.982916 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:08.985591 (+ 2675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:08.986654 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:08.986854 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:08.987034 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:08.987295 (+ 261us) server_negotiation.cc:299] Negotiation successful
0809 15:07:08.987456 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1150,"thread_start_us":1011,"threads_started":1}
I20260809 15:07:08.987862 26840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:08.977368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:44039 (local address 127.0.0.1:40066)
0809 15:07:08.979727 (+ 2359us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:08.979740 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:08.979863 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:08.980785 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:08.980792 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:08.980829 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:07:08.981562 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:08.981575 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:08.983062 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:08.983067 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:08.985434 (+ 2367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:08.985448 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:08.986859 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:08.986866 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:08.987059 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:08.987112 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 15:07:08.987237 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 15:07:08.987418 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2231,"mutex_wait_us":7,"thread_start_us":2112,"threads_started":1}
I20260809 15:07:08.988999 26843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:08.979595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34392 (local address 127.24.183.195:38647)
0809 15:07:08.980447 (+ 852us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:08.980455 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:08.981046 (+ 591us) server_negotiation.cc:407] Connection header received
0809 15:07:08.981516 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:08.981523 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:08.981729 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:08.981901 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:07:08.982805 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:08.983943 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:08.986431 (+ 2488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:08.987789 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:08.987977 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:08.988139 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:08.988497 (+ 358us) server_negotiation.cc:299] Negotiation successful
0809 15:07:08.988653 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":684,"thread_start_us":506,"threads_started":1}
I20260809 15:07:08.989539 26837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:08.976752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:33381 (local address 127.0.0.1:48460)
0809 15:07:08.978448 (+ 1696us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:08.978489 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:08.978672 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:08.981796 (+ 3124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:08.981800 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:08.981828 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:07:08.982285 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:08.982298 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:08.984380 (+ 2082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:08.984389 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:08.986500 (+ 2111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:08.986510 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:08.988716 (+ 2206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:08.988722 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:08.988888 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:08.988923 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 15:07:08.988987 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 15:07:08.989119 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1444,"mutex_wait_us":2,"thread_start_us":1302,"threads_started":1}
I20260809 15:07:08.989892 26842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:08.978946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:38647 (local address 127.0.0.1:34392)
0809 15:07:08.980896 (+ 1950us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:08.980912 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:08.981280 (+ 368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:08.982189 (+ 909us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:08.982193 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:08.982217 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:07:08.982585 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:08.982598 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:08.984073 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:08.984077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:08.986254 (+ 2177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:08.986276 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:08.988224 (+ 1948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:08.988229 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:08.988336 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:08.988358 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 15:07:08.988438 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 15:07:08.989330 (+ 892us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1805,"mutex_wait_us":49,"thread_start_us":1661,"threads_started":1}
I20260809 15:07:08.990676 26839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:08.977017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48460 (local address 127.24.183.194:33381)
0809 15:07:08.981212 (+ 4195us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:08.981220 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:08.981249 (+ 29us) server_negotiation.cc:407] Connection header received
0809 15:07:08.981346 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:08.981354 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:08.981580 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:08.982507 (+ 927us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:07:08.982757 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:08.984247 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:08.986631 (+ 2384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:08.988499 (+ 1868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:08.990145 (+ 1646us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:08.990271 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:08.990321 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 15:07:08.990442 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4052,"thread_start_us":3903,"threads_started":1}
I20260809 15:07:08.992164 26336 tablet_service.cc:1467] Processing CreateTablet for tablet df275e31a0274a7f85660351ae81cd59 (DEFAULT_TABLE table=test-table [id=b87c1c54254346a2892b7dcee3d76874]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:07:08.993784 26336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df275e31a0274a7f85660351ae81cd59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:08.993436 26732 tablet_service.cc:1467] Processing CreateTablet for tablet df275e31a0274a7f85660351ae81cd59 (DEFAULT_TABLE table=test-table [id=b87c1c54254346a2892b7dcee3d76874]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:07:08.993502 26537 tablet_service.cc:1467] Processing CreateTablet for tablet df275e31a0274a7f85660351ae81cd59 (DEFAULT_TABLE table=test-table [id=b87c1c54254346a2892b7dcee3d76874]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:07:08.994802 26732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df275e31a0274a7f85660351ae81cd59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:08.994889 26537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df275e31a0274a7f85660351ae81cd59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:09.005824 26844 tablet_bootstrap.cc:492] T df275e31a0274a7f85660351ae81cd59 P 7cd9ffa189f54c65b7fbbcfdf2437123: Bootstrap starting.
I20260809 15:07:09.008410 26846 tablet_bootstrap.cc:492] T df275e31a0274a7f85660351ae81cd59 P 92b7225b8e97491bab64f4f4fe62b0d7: Bootstrap starting.
I20260809 15:07:09.010401 26844 tablet_bootstrap.cc:654] T df275e31a0274a7f85660351ae81cd59 P 7cd9ffa189f54c65b7fbbcfdf2437123: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:09.011722 26844 log.cc:824] T df275e31a0274a7f85660351ae81cd59 P 7cd9ffa189f54c65b7fbbcfdf2437123: Log is configured to *not* fsync() on all Append() calls
I20260809 15:07:09.012406 26847 tablet_bootstrap.cc:492] T df275e31a0274a7f85660351ae81cd59 P 1dd2a2b141f84eb0acf4e216d5348c2d: Bootstrap starting.
I20260809 15:07:09.012995 26846 tablet_bootstrap.cc:654] T df275e31a0274a7f85660351ae81cd59 P 92b7225b8e97491bab64f4f4fe62b0d7: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:09.014299 26846 log.cc:824] T df275e31a0274a7f85660351ae81cd59 P 92b7225b8e97491bab64f4f4fe62b0d7: Log is configured to *not* fsync() on all Append() calls
I20260809 15:07:09.015380 26847 tablet_bootstrap.cc:654] T df275e31a0274a7f85660351ae81cd59 P 1dd2a2b141f84eb0acf4e216d5348c2d: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:09.016527 26847 log.cc:824] T df275e31a0274a7f85660351ae81cd59 P 1dd2a2b141f84eb0acf4e216d5348c2d: Log is configured to *not* fsync() on all Append() calls
I20260809 15:07:09.018056 26846 tablet_bootstrap.cc:492] T df275e31a0274a7f85660351ae81cd59 P 92b7225b8e97491bab64f4f4fe62b0d7: No bootstrap required, opened a new log
I20260809 15:07:09.018355 26846 ts_tablet_manager.cc:1397] T df275e31a0274a7f85660351ae81cd59 P 92b7225b8e97491bab64f4f4fe62b0d7: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260809 15:07:09.019008 26847 tablet_bootstrap.cc:492] T df275e31a0274a7f85660351ae81cd59 P 1dd2a2b141f84eb0acf4e216d5348c2d: No bootstrap required, opened a new log
I20260809 15:07:09.019239 26847 ts_tablet_manager.cc:1397] T df275e31a0274a7f85660351ae81cd59 P 1dd2a2b141f84eb0acf4e216d5348c2d: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.004s
I20260809 15:07:09.022341 26844 tablet_bootstrap.cc:492] T df275e31a0274a7f85660351ae81cd59 P 7cd9ffa189f54c65b7fbbcfdf2437123: No bootstrap required, opened a new log
I20260809 15:07:09.022608 26844 ts_tablet_manager.cc:1397] T df275e31a0274a7f85660351ae81cd59 P 7cd9ffa189f54c65b7fbbcfdf2437123: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.011s
I20260809 15:07:09.026683 26847 raft_consensus.cc:348] T df275e31a0274a7f85660351ae81cd59 P 1dd2a2b141f84eb0acf4e216d5348c2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd9ffa189f54c65b7fbbcfdf2437123" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 33381 } } peers { permanent_uuid: "1dd2a2b141f84eb0acf4e216d5348c2d" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38647 } } peers { permanent_uuid: "92b7225b8e97491bab64f4f4fe62b0d7" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 44039 } }
I20260809 15:07:09.027107 26847 raft_consensus.cc:374] T df275e31a0274a7f85660351ae81cd59 P 1dd2a2b141f84eb0acf4e216d5348c2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:09.027208 26847 raft_consensus.cc:729] T df275e31a0274a7f85660351ae81cd59 P 1dd2a2b141f84eb0acf4e216d5348c2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dd2a2b141f84eb0acf4e216d5348c2d, State: Initialized, Role: FOLLOWER
I20260809 15:07:09.027845 26847 consensus_queue.cc:260] T df275e31a0274a7f85660351ae81cd59 P 1dd2a2b141f84eb0acf4e216d5348c2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd9ffa189f54c65b7fbbcfdf2437123" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 33381 } } peers { permanent_uuid: "1dd2a2b141f84eb0acf4e216d5348c2d" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38647 } } peers { permanent_uuid: "92b7225b8e97491bab64f4f4fe62b0d7" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 44039 } }
I20260809 15:07:09.028925 26816 heartbeater.cc:503] Master 127.24.183.254:45917 was elected leader, sending a full tablet report...
I20260809 15:07:09.029389 26847 ts_tablet_manager.cc:1428] T df275e31a0274a7f85660351ae81cd59 P 1dd2a2b141f84eb0acf4e216d5348c2d: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260809 15:07:09.028517 26846 raft_consensus.cc:348] T df275e31a0274a7f85660351ae81cd59 P 92b7225b8e97491bab64f4f4fe62b0d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd9ffa189f54c65b7fbbcfdf2437123" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 33381 } } peers { permanent_uuid: "1dd2a2b141f84eb0acf4e216d5348c2d" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38647 } } peers { permanent_uuid: "92b7225b8e97491bab64f4f4fe62b0d7" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 44039 } }
I20260809 15:07:09.030484 26846 raft_consensus.cc:374] T df275e31a0274a7f85660351ae81cd59 P 92b7225b8e97491bab64f4f4fe62b0d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:09.030586 26846 raft_consensus.cc:729] T df275e31a0274a7f85660351ae81cd59 P 92b7225b8e97491bab64f4f4fe62b0d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92b7225b8e97491bab64f4f4fe62b0d7, State: Initialized, Role: FOLLOWER
I20260809 15:07:09.031160 26846 consensus_queue.cc:260] T df275e31a0274a7f85660351ae81cd59 P 92b7225b8e97491bab64f4f4fe62b0d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd9ffa189f54c65b7fbbcfdf2437123" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 33381 } } peers { permanent_uuid: "1dd2a2b141f84eb0acf4e216d5348c2d" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38647 } } peers { permanent_uuid: "92b7225b8e97491bab64f4f4fe62b0d7" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 44039 } }
I20260809 15:07:09.033008 26846 ts_tablet_manager.cc:1428] T df275e31a0274a7f85660351ae81cd59 P 92b7225b8e97491bab64f4f4fe62b0d7: Time spent starting tablet: real 0.015s user 0.009s sys 0.005s
I20260809 15:07:09.033017 26844 raft_consensus.cc:348] T df275e31a0274a7f85660351ae81cd59 P 7cd9ffa189f54c65b7fbbcfdf2437123 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd9ffa189f54c65b7fbbcfdf2437123" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 33381 } } peers { permanent_uuid: "1dd2a2b141f84eb0acf4e216d5348c2d" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38647 } } peers { permanent_uuid: "92b7225b8e97491bab64f4f4fe62b0d7" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 44039 } }
I20260809 15:07:09.033444 26844 raft_consensus.cc:374] T df275e31a0274a7f85660351ae81cd59 P 7cd9ffa189f54c65b7fbbcfdf2437123 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:09.033536 26844 raft_consensus.cc:729] T df275e31a0274a7f85660351ae81cd59 P 7cd9ffa189f54c65b7fbbcfdf2437123 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cd9ffa189f54c65b7fbbcfdf2437123, State: Initialized, Role: FOLLOWER
I20260809 15:07:09.034102 26844 consensus_queue.cc:260] T df275e31a0274a7f85660351ae81cd59 P 7cd9ffa189f54c65b7fbbcfdf2437123 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd9ffa189f54c65b7fbbcfdf2437123" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 33381 } } peers { permanent_uuid: "1dd2a2b141f84eb0acf4e216d5348c2d" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38647 } } peers { permanent_uuid: "92b7225b8e97491bab64f4f4fe62b0d7" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 44039 } }
I20260809 15:07:09.035734 26627 heartbeater.cc:503] Master 127.24.183.254:45917 was elected leader, sending a full tablet report...
I20260809 15:07:09.038007 26844 ts_tablet_manager.cc:1428] T df275e31a0274a7f85660351ae81cd59 P 7cd9ffa189f54c65b7fbbcfdf2437123: Time spent starting tablet: real 0.015s user 0.013s sys 0.002s
W20260809 15:07:09.145879 26438 tablet.cc:2369] T df275e31a0274a7f85660351ae81cd59 P 92b7225b8e97491bab64f4f4fe62b0d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:07:09.149681 26629 tablet.cc:2369] T df275e31a0274a7f85660351ae81cd59 P 7cd9ffa189f54c65b7fbbcfdf2437123: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:07:09.150813 26817 tablet.cc:2369] T df275e31a0274a7f85660351ae81cd59 P 1dd2a2b141f84eb0acf4e216d5348c2d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:07:09.153846 26853 raft_consensus.cc:482] T df275e31a0274a7f85660351ae81cd59 P 92b7225b8e97491bab64f4f4fe62b0d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:07:09.154104 26853 raft_consensus.cc:504] T df275e31a0274a7f85660351ae81cd59 P 92b7225b8e97491bab64f4f4fe62b0d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd9ffa189f54c65b7fbbcfdf2437123" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 33381 } } peers { permanent_uuid: "1dd2a2b141f84eb0acf4e216d5348c2d" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38647 } } peers { permanent_uuid: "92b7225b8e97491bab64f4f4fe62b0d7" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 44039 } }
I20260809 15:07:09.155880 26853 leader_election.cc:288] T df275e31a0274a7f85660351ae81cd59 P 92b7225b8e97491bab64f4f4fe62b0d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cd9ffa189f54c65b7fbbcfdf2437123 (127.24.183.194:33381), 1dd2a2b141f84eb0acf4e216d5348c2d (127.24.183.195:38647)
I20260809 15:07:09.163199 26839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:09.156384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:52747 (local address 127.24.183.194:33381)
0809 15:07:09.156636 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:09.156645 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:09.157139 (+ 494us) server_negotiation.cc:407] Connection header received
0809 15:07:09.157329 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:09.157334 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:09.157421 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:09.157512 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:07:09.158249 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:09.159161 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:09.161256 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:09.162200 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:09.162312 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:09.162368 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:09.162840 (+ 472us) server_negotiation.cc:299] Negotiation successful
0809 15:07:09.162934 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 15:07:09.163316 26857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:09.156338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:33381 (local address 127.24.183.193:52747)
0809 15:07:09.157024 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:09.157045 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:09.157197 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:09.157656 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:09.157662 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:09.157698 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:07:09.158096 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:09.158110 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:09.159293 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:09.159300 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:09.161096 (+ 1796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:09.161110 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:09.162429 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:09.162436 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:09.162632 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:09.162675 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 15:07:09.162820 (+ 145us) client_negotiation.cc:240] Negotiation successful
0809 15:07:09.162946 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":359,"threads_started":1}
I20260809 15:07:09.163885 26843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:09.157324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:38263 (local address 127.24.183.195:38647)
0809 15:07:09.157548 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:09.157554 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:09.157893 (+ 339us) server_negotiation.cc:407] Connection header received
0809 15:07:09.158022 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:09.158028 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:09.158111 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:09.158220 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:07:09.158851 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:09.159926 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:09.161786 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:09.162868 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:09.163009 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:09.163082 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:09.163491 (+ 409us) server_negotiation.cc:299] Negotiation successful
0809 15:07:09.163628 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 15:07:09.164381 26860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:09.157180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:38647 (local address 127.24.183.193:38263)
0809 15:07:09.157785 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:09.157798 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:09.157902 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:09.158238 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:09.158242 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:09.158277 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:07:09.158684 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:09.158696 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:09.160064 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:09.160067 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:09.161586 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:09.161601 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:09.163103 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:09.163109 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:09.163285 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:09.163323 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 15:07:09.163983 (+ 660us) client_negotiation.cc:240] Negotiation successful
0809 15:07:09.164111 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":289,"threads_started":1}
I20260809 15:07:09.164474 26563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df275e31a0274a7f85660351ae81cd59" candidate_uuid: "92b7225b8e97491bab64f4f4fe62b0d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cd9ffa189f54c65b7fbbcfdf2437123" is_pre_election: true
I20260809 15:07:09.165074 26563 raft_consensus.cc:2454] T df275e31a0274a7f85660351ae81cd59 P 7cd9ffa189f54c65b7fbbcfdf2437123 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92b7225b8e97491bab64f4f4fe62b0d7 in term 0.
I20260809 15:07:09.165321 26757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df275e31a0274a7f85660351ae81cd59" candidate_uuid: "92b7225b8e97491bab64f4f4fe62b0d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dd2a2b141f84eb0acf4e216d5348c2d" is_pre_election: true
I20260809 15:07:09.165793 26271 leader_election.cc:302] T df275e31a0274a7f85660351ae81cd59 P 92b7225b8e97491bab64f4f4fe62b0d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cd9ffa189f54c65b7fbbcfdf2437123, 92b7225b8e97491bab64f4f4fe62b0d7; no voters:
I20260809 15:07:09.165866 26757 raft_consensus.cc:2454] T df275e31a0274a7f85660351ae81cd59 P 1dd2a2b141f84eb0acf4e216d5348c2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92b7225b8e97491bab64f4f4fe62b0d7 in term 0.
I20260809 15:07:09.166244 26853 raft_consensus.cc:2781] T df275e31a0274a7f85660351ae81cd59 P 92b7225b8e97491bab64f4f4fe62b0d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:07:09.166392 26853 raft_consensus.cc:482] T df275e31a0274a7f85660351ae81cd59 P 92b7225b8e97491bab64f4f4fe62b0d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:07:09.166507 26853 raft_consensus.cc:3037] T df275e31a0274a7f85660351ae81cd59 P 92b7225b8e97491bab64f4f4fe62b0d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:09.168877 26853 raft_consensus.cc:504] T df275e31a0274a7f85660351ae81cd59 P 92b7225b8e97491bab64f4f4fe62b0d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd9ffa189f54c65b7fbbcfdf2437123" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 33381 } } peers { permanent_uuid: "1dd2a2b141f84eb0acf4e216d5348c2d" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38647 } } peers { permanent_uuid: "92b7225b8e97491bab64f4f4fe62b0d7" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 44039 } }
I20260809 15:07:09.169592 26853 leader_election.cc:288] T df275e31a0274a7f85660351ae81cd59 P 92b7225b8e97491bab64f4f4fe62b0d7 [CANDIDATE]: Term 1 election: Requested vote from peers 7cd9ffa189f54c65b7fbbcfdf2437123 (127.24.183.194:33381), 1dd2a2b141f84eb0acf4e216d5348c2d (127.24.183.195:38647)
I20260809 15:07:09.169997 26563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df275e31a0274a7f85660351ae81cd59" candidate_uuid: "92b7225b8e97491bab64f4f4fe62b0d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cd9ffa189f54c65b7fbbcfdf2437123"
I20260809 15:07:09.170079 26757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df275e31a0274a7f85660351ae81cd59" candidate_uuid: "92b7225b8e97491bab64f4f4fe62b0d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dd2a2b141f84eb0acf4e216d5348c2d"
I20260809 15:07:09.170187 26563 raft_consensus.cc:3037] T df275e31a0274a7f85660351ae81cd59 P 7cd9ffa189f54c65b7fbbcfdf2437123 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:09.170260 26757 raft_consensus.cc:3037] T df275e31a0274a7f85660351ae81cd59 P 1dd2a2b141f84eb0acf4e216d5348c2d [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:09.172446 26563 raft_consensus.cc:2454] T df275e31a0274a7f85660351ae81cd59 P 7cd9ffa189f54c65b7fbbcfdf2437123 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92b7225b8e97491bab64f4f4fe62b0d7 in term 1.
I20260809 15:07:09.172451 26757 raft_consensus.cc:2454] T df275e31a0274a7f85660351ae81cd59 P 1dd2a2b141f84eb0acf4e216d5348c2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92b7225b8e97491bab64f4f4fe62b0d7 in term 1.
I20260809 15:07:09.172799 26271 leader_election.cc:302] T df275e31a0274a7f85660351ae81cd59 P 92b7225b8e97491bab64f4f4fe62b0d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cd9ffa189f54c65b7fbbcfdf2437123, 92b7225b8e97491bab64f4f4fe62b0d7; no voters:
I20260809 15:07:09.173044 26853 raft_consensus.cc:2781] T df275e31a0274a7f85660351ae81cd59 P 92b7225b8e97491bab64f4f4fe62b0d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:07:09.173465 26853 raft_consensus.cc:686] T df275e31a0274a7f85660351ae81cd59 P 92b7225b8e97491bab64f4f4fe62b0d7 [term 1 LEADER]: Becoming Leader. State: Replica: 92b7225b8e97491bab64f4f4fe62b0d7, State: Running, Role: LEADER
I20260809 15:07:09.174003 26853 consensus_queue.cc:237] T df275e31a0274a7f85660351ae81cd59 P 92b7225b8e97491bab64f4f4fe62b0d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd9ffa189f54c65b7fbbcfdf2437123" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 33381 } } peers { permanent_uuid: "1dd2a2b141f84eb0acf4e216d5348c2d" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38647 } } peers { permanent_uuid: "92b7225b8e97491bab64f4f4fe62b0d7" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 44039 } }
I20260809 15:07:09.178382 26144 catalog_manager.cc:5495] T df275e31a0274a7f85660351ae81cd59 P 92b7225b8e97491bab64f4f4fe62b0d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 92b7225b8e97491bab64f4f4fe62b0d7 (127.24.183.193). New cstate: current_term: 1 leader_uuid: "92b7225b8e97491bab64f4f4fe62b0d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd9ffa189f54c65b7fbbcfdf2437123" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 33381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dd2a2b141f84eb0acf4e216d5348c2d" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92b7225b8e97491bab64f4f4fe62b0d7" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 44039 } health_report { overall_health: HEALTHY } } }
Aug 09 15:07:09 dist-test-slave-nj21 krb5kdc[26076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.193@KRBTEST.COM
I20260809 15:07:09.228786 26841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:09.214787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40074 (local address 127.24.183.193:44039)
0809 15:07:09.215043 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:09.215051 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:09.215503 (+ 452us) server_negotiation.cc:407] Connection header received
0809 15:07:09.215637 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:09.215642 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:09.215738 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:09.215826 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:09.217322 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:09.218333 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:09.219546 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:09.219996 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:09.220097 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:09.223804 (+ 3707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:09.223830 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:09.223849 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:09.223889 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:09.226492 (+ 2603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:09.227061 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:09.227067 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:09.227070 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:09.227228 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:09.227496 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:09.227498 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:09.227500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:09.227901 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:09.228138 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:09.228345 (+ 207us) server_negotiation.cc:299] Negotiation successful
0809 15:07:09.228520 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
Aug 09 15:07:09 dist-test-slave-nj21 krb5kdc[26076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288029, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 15:07:09 dist-test-slave-nj21 krb5kdc[26076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288029, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.183.193@KRBTEST.COM
I20260809 15:07:09.277287 26841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:09.264031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40090 (local address 127.24.183.193:44039)
0809 15:07:09.264295 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:09.264304 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:09.264661 (+ 357us) server_negotiation.cc:407] Connection header received
0809 15:07:09.264805 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:09.264810 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:09.264888 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:09.264977 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:09.266366 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:09.267114 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:09.268462 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:09.268882 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:09.268976 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:09.272735 (+ 3759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:09.272740 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:09.272744 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:09.272777 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:09.275196 (+ 2419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:09.275696 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:09.275701 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:09.275704 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:09.275837 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:09.276158 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:09.276161 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:09.276163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:09.276410 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:09.276626 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:09.276786 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 15:07:09.276953 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
Aug 09 15:07:09 dist-test-slave-nj21 krb5kdc[26076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 15:07:09 dist-test-slave-nj21 krb5kdc[26076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.193@KRBTEST.COM
I20260809 15:07:09.323838 26841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:09.310495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40092 (local address 127.24.183.193:44039)
0809 15:07:09.310696 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:09.310704 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:09.311027 (+ 323us) server_negotiation.cc:407] Connection header received
0809 15:07:09.311207 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:09.311213 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:09.311298 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:09.311395 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:09.312800 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:09.313585 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:09.314784 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:09.315216 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:09.315337 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:09.318987 (+ 3650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:09.318993 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:09.318996 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:09.319028 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:09.321411 (+ 2383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:09.322004 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:09.322008 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:09.322010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:09.322134 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:09.322462 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:09.322467 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:09.322470 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:09.322728 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:09.322972 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:09.323254 (+ 282us) server_negotiation.cc:299] Negotiation successful
0809 15:07:09.323477 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260809 15:07:09.325699 26307 tablet_service.cc:2908] Rejecting scan request for tablet df275e31a0274a7f85660351ae81cd59: Uninitialized: clean time has not yet been initialized
I20260809 15:07:09.328881 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 26245
I20260809 15:07:09.341395 26841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:09.329890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40098 (local address 127.24.183.193:44039)
0809 15:07:09.330051 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:09.330056 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:09.330191 (+ 135us) server_negotiation.cc:407] Connection header received
0809 15:07:09.330318 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:09.330323 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:09.330390 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:09.330473 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:09.331577 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:09.332660 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:09.333839 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:09.334292 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:09.334411 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:09.336542 (+ 2131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:09.336547 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:09.336550 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:09.336579 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:09.338979 (+ 2400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:09.339582 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:09.339587 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:09.339589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:09.339730 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:09.340066 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:09.340072 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:09.340075 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:09.340337 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:09.340562 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:09.340850 (+ 288us) server_negotiation.cc:299] Negotiation successful
0809 15:07:09.341077 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 15:07:09.341971 26424 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:07:09.438673 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 26245
I20260809 15:07:09.463521 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 26443
Aug 09 15:07:09 dist-test-slave-nj21 krb5kdc[26076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.194@KRBTEST.COM
I20260809 15:07:09.477991 26839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:09.464331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48472 (local address 127.24.183.194:33381)
0809 15:07:09.464531 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:09.464536 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:09.464556 (+ 20us) server_negotiation.cc:407] Connection header received
0809 15:07:09.464673 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:09.464677 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:09.464798 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:09.464883 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:09.466018 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:09.466806 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:09.468139 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:09.468563 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:09.468681 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:09.472868 (+ 4187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:09.472882 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:09.472900 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:09.472938 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:09.475476 (+ 2538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:09.476211 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:09.476219 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:09.476221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:09.476352 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:09.476684 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:09.476686 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:09.476688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:09.477058 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:09.477307 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:09.477548 (+ 241us) server_negotiation.cc:299] Negotiation successful
0809 15:07:09.477714 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 15:07:09.478601 26618 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:07:09.566960 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 26443
I20260809 15:07:09.590096 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 26635
Aug 09 15:07:09 dist-test-slave-nj21 krb5kdc[26076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.195@KRBTEST.COM
I20260809 15:07:09.604900 26843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:09.590881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34400 (local address 127.24.183.195:38647)
0809 15:07:09.591044 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:09.591048 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:09.591068 (+ 20us) server_negotiation.cc:407] Connection header received
0809 15:07:09.591185 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:09.591190 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:09.591255 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:09.591347 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:09.592705 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:09.593448 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:09.594789 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:09.595199 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:09.595289 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:09.599550 (+ 4261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:09.599566 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:09.599585 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:09.599619 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:09.602314 (+ 2695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:09.602963 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:09.602970 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:09.602972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:09.603096 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:09.603408 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:09.603411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:09.603413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:09.603819 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:09.604052 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:09.604343 (+ 291us) server_negotiation.cc:299] Negotiation successful
0809 15:07:09.604513 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 15:07:09.605492 26809 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:07:09.696077 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 26635
I20260809 15:07:09.717931 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 26097
I20260809 15:07:09.719144 26200 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:07:09.829457 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 26097
2026-08-09T15:07:09Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4308 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:07:09 dist-test-slave-nj21 krb5kdc[26887](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:07:09 dist-test-slave-nj21 krb5kdc[26887](info): set up 2 sockets
Aug 09 15:07:09 dist-test-slave-nj21 krb5kdc[26887](info): commencing operation
krb5kdc: starting...
W20260809 15:07:11.020798 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.001s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:07:11 dist-test-slave-nj21 krb5kdc[26887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:07:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:07:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
I20260809 15:07:11.200565 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:42375
--webserver_interface=127.24.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:33567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:42375
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:11.521292 26902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:11.521569 26902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:11.521667 26902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:11.528680 26902 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:07:11.528779 26902 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 15:07:11.528826 26902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:11.528887 26902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:11.528935 26902 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:07:11.528997 26902 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:07:11.538563 26902 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:33567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:42375
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:42375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:11.539760 26902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:11.541002 26902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:11.552277 26907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:11.552242 26908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:11.552392 26911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:11.553292 26902 server_base.cc:1034] running on GCE node
I20260809 15:07:11.554024 26902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:11.555348 26902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:11.556560 26902 hybrid_clock.cc:648] HybridClock initialized: now 1786288031556496 us; error 38 us; skew 500 ppm
I20260809 15:07:11.556885 26902 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:11 dist-test-slave-nj21 krb5kdc[26887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288031, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:11.559814 26902 init.cc:377] Logged in from keytab as kudu/127.24.183.254@KRBTEST.COM (short username kudu)
I20260809 15:07:11.561264 26902 webserver.cc:460] Webserver started at http://127.24.183.254:41571/ using document root <none> and password file <none>
I20260809 15:07:11.561939 26902 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:11.562038 26902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:11.562350 26902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:11.565259 26902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "a7c251fa7fa546fc86a1d5e92878800e"
format_stamp: "Formatted at 2026-08-09 15:07:11 on dist-test-slave-nj21"
server_key: "8a067a7b49dea64ff7368089b0d89455"
server_key_iv: "179cc0442faff681bf488e8a77f2c719"
server_key_version: "encryptionkey@0"
I20260809 15:07:11.566030 26902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "a7c251fa7fa546fc86a1d5e92878800e"
format_stamp: "Formatted at 2026-08-09 15:07:11 on dist-test-slave-nj21"
server_key: "8a067a7b49dea64ff7368089b0d89455"
server_key_iv: "179cc0442faff681bf488e8a77f2c719"
server_key_version: "encryptionkey@0"
I20260809 15:07:11.570927 26902 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 15:07:11.574275 26922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:11.575572 26902 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:07:11.575773 26902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "a7c251fa7fa546fc86a1d5e92878800e"
format_stamp: "Formatted at 2026-08-09 15:07:11 on dist-test-slave-nj21"
server_key: "8a067a7b49dea64ff7368089b0d89455"
server_key_iv: "179cc0442faff681bf488e8a77f2c719"
server_key_version: "encryptionkey@0"
I20260809 15:07:11.575968 26902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:11.594141 26902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:11.597879 26902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:11.598127 26902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:11.618815 26902 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:42375
I20260809 15:07:11.618839 27010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:42375 every 8 connection(s)
I20260809 15:07:11.620144 26902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:07:11.623780 27011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:11.625524 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 26902
I20260809 15:07:11.625737 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:07:11.626392 25311 external_mini_cluster.cc:1428] Setting key a02c505163f48c65dd1caaa39af2be7f
I20260809 15:07:11.635319 27011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7c251fa7fa546fc86a1d5e92878800e: Bootstrap starting.
I20260809 15:07:11.639101 27011 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7c251fa7fa546fc86a1d5e92878800e: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:11.640336 27011 log.cc:824] T 00000000000000000000000000000000 P a7c251fa7fa546fc86a1d5e92878800e: Log is configured to *not* fsync() on all Append() calls
Aug 09 15:07:11 dist-test-slave-nj21 krb5kdc[26887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:11.643033 27011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7c251fa7fa546fc86a1d5e92878800e: No bootstrap required, opened a new log
I20260809 15:07:11.649668 27015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:11.628257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59834 (local address 127.24.183.254:42375)
0809 15:07:11.629089 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:11.629100 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:11.629141 (+ 41us) server_negotiation.cc:407] Connection header received
0809 15:07:11.630842 (+ 1701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:11.630865 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:11.631212 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:11.631530 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:11.633231 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:11.634472 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:11.636126 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:11.636778 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:11.636915 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:11.641704 (+ 4789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:11.641712 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:11.641726 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:11.641762 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:11.645315 (+ 3553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:11.645953 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:11.645962 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:11.645970 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:11.646191 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:11.646516 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:11.646520 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:11.646527 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:11.647241 (+ 714us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:11.647529 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:11.647825 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 15:07:11.648272 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":387,"threads_started":1}
I20260809 15:07:11.652586 27011 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7c251fa7fa546fc86a1d5e92878800e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7c251fa7fa546fc86a1d5e92878800e" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 42375 } }
I20260809 15:07:11.652895 27011 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7c251fa7fa546fc86a1d5e92878800e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:11.652968 27011 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7c251fa7fa546fc86a1d5e92878800e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7c251fa7fa546fc86a1d5e92878800e, State: Initialized, Role: FOLLOWER
I20260809 15:07:11.653434 27011 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7c251fa7fa546fc86a1d5e92878800e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7c251fa7fa546fc86a1d5e92878800e" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 42375 } }
I20260809 15:07:11.653633 27011 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7c251fa7fa546fc86a1d5e92878800e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:07:11.653730 27011 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7c251fa7fa546fc86a1d5e92878800e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:07:11.653849 27011 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7c251fa7fa546fc86a1d5e92878800e [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:11.656090 27011 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7c251fa7fa546fc86a1d5e92878800e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7c251fa7fa546fc86a1d5e92878800e" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 42375 } }
I20260809 15:07:11.656539 27011 leader_election.cc:302] T 00000000000000000000000000000000 P a7c251fa7fa546fc86a1d5e92878800e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7c251fa7fa546fc86a1d5e92878800e; no voters:
I20260809 15:07:11.657013 27011 leader_election.cc:288] T 00000000000000000000000000000000 P a7c251fa7fa546fc86a1d5e92878800e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:07:11.657143 27018 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7c251fa7fa546fc86a1d5e92878800e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:07:11.657809 27018 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7c251fa7fa546fc86a1d5e92878800e [term 1 LEADER]: Becoming Leader. State: Replica: a7c251fa7fa546fc86a1d5e92878800e, State: Running, Role: LEADER
I20260809 15:07:11.658628 27018 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7c251fa7fa546fc86a1d5e92878800e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7c251fa7fa546fc86a1d5e92878800e" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 42375 } }
I20260809 15:07:11.659063 27011 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7c251fa7fa546fc86a1d5e92878800e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:07:11.661381 27021 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7c251fa7fa546fc86a1d5e92878800e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7c251fa7fa546fc86a1d5e92878800e. Latest consensus state: current_term: 1 leader_uuid: "a7c251fa7fa546fc86a1d5e92878800e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7c251fa7fa546fc86a1d5e92878800e" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 42375 } } }
I20260809 15:07:11.661350 27019 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7c251fa7fa546fc86a1d5e92878800e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7c251fa7fa546fc86a1d5e92878800e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7c251fa7fa546fc86a1d5e92878800e" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 42375 } } }
I20260809 15:07:11.661788 27021 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7c251fa7fa546fc86a1d5e92878800e [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:11.661789 27019 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7c251fa7fa546fc86a1d5e92878800e [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:11.663095 27025 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:07:11.668551 27025 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:07:11.677549 27025 catalog_manager.cc:1347] Generated new cluster ID: 7a0a541382f2443a89c140e538975933
I20260809 15:07:11.677660 27025 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:07:11.694566 27025 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:07:11.695889 27025 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:07:11.712999 27025 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7c251fa7fa546fc86a1d5e92878800e: Generated new TSK 0
I20260809 15:07:11.713768 27025 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab.
Entry for principal kudu/127.24.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab.
I20260809 15:07:11.771216 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.193:0
--local_ip_for_outbound_sockets=127.24.183.193
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:42375
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:33567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:12.132035 27049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:12.132313 27049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:12.132422 27049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:12.139802 27049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:12.139914 27049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:12.140041 27049 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 15:07:12.140143 27049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.193
I20260809 15:07:12.150300 27049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:33567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.183.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.183.254:42375
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.193
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:12.151731 27049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:12.153115 27049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:12.165856 27057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:12.165899 27056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:12.165840 27059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:12.166831 27049 server_base.cc:1034] running on GCE node
I20260809 15:07:12.167414 27049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:12.168316 27049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:12.169526 27049 hybrid_clock.cc:648] HybridClock initialized: now 1786288032169455 us; error 43 us; skew 500 ppm
I20260809 15:07:12.169780 27049 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:12 dist-test-slave-nj21 krb5kdc[26887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288032, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:12.173075 27049 init.cc:377] Logged in from keytab as kudu/127.24.183.193@KRBTEST.COM (short username kudu)
I20260809 15:07:12.174490 27049 webserver.cc:460] Webserver started at http://127.24.183.193:36847/ using document root <none> and password file <none>
I20260809 15:07:12.175137 27049 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:12.175225 27049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:12.175508 27049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:12.178396 27049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-0/data/instance:
uuid: "be36dd1898cd4b008b8295d3923410bd"
format_stamp: "Formatted at 2026-08-09 15:07:12 on dist-test-slave-nj21"
server_key: "22728df58c409671feb49be0778b9f8d"
server_key_iv: "36745ba629d9155b62350314082891b6"
server_key_version: "encryptionkey@0"
I20260809 15:07:12.179188 27049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance:
uuid: "be36dd1898cd4b008b8295d3923410bd"
format_stamp: "Formatted at 2026-08-09 15:07:12 on dist-test-slave-nj21"
server_key: "22728df58c409671feb49be0778b9f8d"
server_key_iv: "36745ba629d9155b62350314082891b6"
server_key_version: "encryptionkey@0"
I20260809 15:07:12.184262 27049 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260809 15:07:12.187551 27072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:12.188812 27049 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:07:12.188944 27049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-0/wal
uuid: "be36dd1898cd4b008b8295d3923410bd"
format_stamp: "Formatted at 2026-08-09 15:07:12 on dist-test-slave-nj21"
server_key: "22728df58c409671feb49be0778b9f8d"
server_key_iv: "36745ba629d9155b62350314082891b6"
server_key_version: "encryptionkey@0"
I20260809 15:07:12.189123 27049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:12.215181 27049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:12.219350 27049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:12.219585 27049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:12.222378 27049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:07:12.222481 27049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:12.222607 27049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:07:12.222681 27049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 15:07:12 dist-test-slave-nj21 krb5kdc[26887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288032, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:12.245710 27083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:12.222970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:42303 (local address 127.24.183.254:42375)
0809 15:07:12.224022 (+ 1052us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:12.224026 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:12.224697 (+ 671us) server_negotiation.cc:407] Connection header received
0809 15:07:12.226706 (+ 2009us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:12.226712 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:12.226799 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:12.226896 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:12.229016 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:12.230019 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:12.232436 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:12.233108 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:12.233280 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:12.238326 (+ 5046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:12.238344 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:12.238349 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:12.238396 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:12.241480 (+ 3084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:12.242516 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:12.242522 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:12.242526 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:12.242691 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:12.243270 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:12.243275 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:12.243279 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:12.243619 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:12.243804 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:12.245073 (+ 1269us) server_negotiation.cc:299] Negotiation successful
0809 15:07:12.245334 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":941,"thread_start_us":832,"threads_started":1}
I20260809 15:07:12.247516 27084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:12.223750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:42375 (local address 127.24.183.193:42303)
0809 15:07:12.224555 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:12.224594 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:12.226429 (+ 1835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:12.227150 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:12.227157 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:12.227890 (+ 733us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:12.228806 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:12.228847 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:12.230174 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:12.230178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:12.232272 (+ 2094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:12.232291 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:12.233371 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:12.233377 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:12.233565 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:12.234272 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:12.234298 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:12.237977 (+ 3679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:12.241672 (+ 3695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:12.241682 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:12.241699 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:12.242315 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:12.242802 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:12.242811 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:12.242818 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:12.243110 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:12.243793 (+ 683us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:12.243805 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:12.244483 (+ 678us) client_negotiation.cc:769] Sending connection context
0809 15:07:12.244939 (+ 456us) client_negotiation.cc:240] Negotiation successful
0809 15:07:12.245670 (+ 731us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":346,"threads_started":1}
I20260809 15:07:12.269515 27083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:12.254040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:52685 (local address 127.24.183.254:42375)
0809 15:07:12.254262 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:12.254270 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:12.255161 (+ 891us) server_negotiation.cc:407] Connection header received
0809 15:07:12.255287 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:12.255294 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:12.255394 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:12.255513 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:12.256719 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:12.257430 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:12.258709 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:12.259144 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:12.259251 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:12.263583 (+ 4332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:12.263588 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:12.263592 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:12.263623 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:12.266654 (+ 3031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:12.267466 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:12.267472 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:12.267475 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:12.267633 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:12.268145 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:12.268150 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:12.268154 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:12.268438 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:12.268554 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:12.269058 (+ 504us) server_negotiation.cc:299] Negotiation successful
0809 15:07:12.269224 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 15:07:12.270025 27207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:12.253962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:42375 (local address 127.24.183.193:52685)
0809 15:07:12.255052 (+ 1090us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:12.255068 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:12.255188 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:12.255562 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:12.255567 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:12.256129 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:12.256573 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:12.256588 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:12.257562 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:12.257566 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:12.258578 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:12.258590 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:12.259321 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:12.259327 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:12.259491 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:12.262424 (+ 2933us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:12.262449 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:12.263403 (+ 954us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:12.266775 (+ 3372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:12.266780 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:12.266785 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:12.267360 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:12.267778 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:12.267784 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:12.267788 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:12.268000 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:12.268549 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:12.268556 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:12.268805 (+ 249us) client_negotiation.cc:769] Sending connection context
0809 15:07:12.268979 (+ 174us) client_negotiation.cc:240] Negotiation successful
0809 15:07:12.269402 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":960,"thread_start_us":676,"threads_started":1}
I20260809 15:07:12.278340 27049 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.193:40361
I20260809 15:07:12.278393 27251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.193:40361 every 8 connection(s)
I20260809 15:07:12.279659 27049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
I20260809 15:07:12.281684 27252 heartbeater.cc:347] Connected to a master server at 127.24.183.254:42375
I20260809 15:07:12.281903 27252 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:12.282531 27252 heartbeater.cc:511] Master 127.24.183.254:42375 requested a full tablet report, sending...
I20260809 15:07:12.284430 26943 ts_manager.cc:194] Registered new tserver with Master: be36dd1898cd4b008b8295d3923410bd (127.24.183.193:40361)
I20260809 15:07:12.285876 26943 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.193@KRBTEST.COM'} at 127.24.183.193:42303
I20260809 15:07:12.290089 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 27049
I20260809 15:07:12.290287 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance
I20260809 15:07:12.290921 25311 external_mini_cluster.cc:1428] Setting key 0858a7dfa66abc5bd49eb1ca5da1b5a7
WARNING: no policy specified for kudu/127.24.183.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab.
Entry for principal kudu/127.24.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab.
I20260809 15:07:12.330809 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.194:0
--local_ip_for_outbound_sockets=127.24.183.194
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:42375
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:33567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:12.652840 27259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:12.653093 27259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:12.653227 27259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:12.660440 27259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:12.660552 27259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:12.660663 27259 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 15:07:12.660750 27259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.194
I20260809 15:07:12.670611 27259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:33567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.183.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.183.254:42375
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.194
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:12.671864 27259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:12.673182 27259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:12.686367 27270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:12.686426 27272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:12.686462 27275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:12.687625 27259 server_base.cc:1034] running on GCE node
I20260809 15:07:12.688261 27259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:12.689167 27259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:12.690387 27259 hybrid_clock.cc:648] HybridClock initialized: now 1786288032690322 us; error 32 us; skew 500 ppm
I20260809 15:07:12.690673 27259 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:12 dist-test-slave-nj21 krb5kdc[26887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288032, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:12.694020 27259 init.cc:377] Logged in from keytab as kudu/127.24.183.194@KRBTEST.COM (short username kudu)
I20260809 15:07:12.695489 27259 webserver.cc:460] Webserver started at http://127.24.183.194:41355/ using document root <none> and password file <none>
I20260809 15:07:12.696254 27259 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:12.696357 27259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:12.696659 27259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:12.699904 27259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-1/data/instance:
uuid: "ba2c864293ad499f8e13bee4f688cbd1"
format_stamp: "Formatted at 2026-08-09 15:07:12 on dist-test-slave-nj21"
server_key: "2312069f9693f14ffeef15c3ae996f07"
server_key_iv: "82456e07dce157d0775d3bca78eb0004"
server_key_version: "encryptionkey@0"
I20260809 15:07:12.700742 27259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance:
uuid: "ba2c864293ad499f8e13bee4f688cbd1"
format_stamp: "Formatted at 2026-08-09 15:07:12 on dist-test-slave-nj21"
server_key: "2312069f9693f14ffeef15c3ae996f07"
server_key_iv: "82456e07dce157d0775d3bca78eb0004"
server_key_version: "encryptionkey@0"
I20260809 15:07:12.705986 27259 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 15:07:12.709391 27287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:12.710677 27259 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 15:07:12.710829 27259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-1/wal
uuid: "ba2c864293ad499f8e13bee4f688cbd1"
format_stamp: "Formatted at 2026-08-09 15:07:12 on dist-test-slave-nj21"
server_key: "2312069f9693f14ffeef15c3ae996f07"
server_key_iv: "82456e07dce157d0775d3bca78eb0004"
server_key_version: "encryptionkey@0"
I20260809 15:07:12.711032 27259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:12.726331 27259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:12.730495 27259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:12.730749 27259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:12.733429 27259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:07:12.733536 27259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:12.733667 27259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:07:12.733747 27259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 15:07:12 dist-test-slave-nj21 krb5kdc[26887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288032, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:12.755601 27083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:12.734119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:53577 (local address 127.24.183.254:42375)
0809 15:07:12.734294 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:12.734299 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:12.735847 (+ 1548us) server_negotiation.cc:407] Connection header received
0809 15:07:12.738036 (+ 2189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:12.738042 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:12.738128 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:12.738217 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:12.739694 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:12.740725 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:12.742341 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:12.742899 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:12.743033 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:12.748229 (+ 5196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:12.748236 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:12.748241 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:12.748289 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:12.752004 (+ 3715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:12.753104 (+ 1100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:12.753110 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:12.753114 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:12.753286 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:12.753817 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:12.753823 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:12.753827 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:12.754120 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:12.754272 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:12.754970 (+ 698us) server_negotiation.cc:299] Negotiation successful
0809 15:07:12.755204 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 15:07:12.756743 27300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:12.734795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:42375 (local address 127.24.183.194:53577)
0809 15:07:12.735665 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:12.735740 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:12.737762 (+ 2022us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:12.738464 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:12.738472 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:12.738956 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:12.739501 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:12.739518 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:12.740889 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:12.740892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:12.742189 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:12.742202 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:12.743170 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:12.743179 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:12.743523 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:12.744394 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:12.744441 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:12.747968 (+ 3527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:12.752144 (+ 4176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:12.752153 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:12.752165 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:12.752960 (+ 795us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:12.753422 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:12.753426 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:12.753433 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:12.753698 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:12.754248 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:12.754256 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:12.754610 (+ 354us) client_negotiation.cc:769] Sending connection context
0809 15:07:12.754852 (+ 242us) client_negotiation.cc:240] Negotiation successful
0809 15:07:12.755367 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":354,"threads_started":1}
I20260809 15:07:12.782225 27259 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.194:41627
I20260809 15:07:12.782282 27462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.194:41627 every 8 connection(s)
I20260809 15:07:12.783812 27259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
I20260809 15:07:12.786475 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 27259
I20260809 15:07:12.786664 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance
I20260809 15:07:12.787137 25311 external_mini_cluster.cc:1428] Setting key 09382cb5bcb9db65d4c53fe984b3452d
I20260809 15:07:12.787884 27463 heartbeater.cc:347] Connected to a master server at 127.24.183.254:42375
I20260809 15:07:12.788143 27463 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:12.789054 27463 heartbeater.cc:511] Master 127.24.183.254:42375 requested a full tablet report, sending...
I20260809 15:07:12.790639 26943 ts_manager.cc:194] Registered new tserver with Master: ba2c864293ad499f8e13bee4f688cbd1 (127.24.183.194:41627)
I20260809 15:07:12.791637 27083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:12.777430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:52291 (local address 127.24.183.254:42375)
0809 15:07:12.777610 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:12.777614 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:12.778235 (+ 621us) server_negotiation.cc:407] Connection header received
0809 15:07:12.778435 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:12.778439 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:12.778520 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:12.778617 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:12.780137 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:12.780923 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:12.782498 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:12.783121 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:12.783268 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:12.785493 (+ 2225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:12.785500 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:12.785504 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:12.785547 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:12.788706 (+ 3159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:12.789463 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:12.789469 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:12.789473 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:12.789612 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:12.789973 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:12.789978 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:12.789982 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:12.790437 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:12.790581 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:12.791071 (+ 490us) server_negotiation.cc:299] Negotiation successful
0809 15:07:12.791298 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 15:07:12.791707 27449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:12.777496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:42375 (local address 127.24.183.194:52291)
0809 15:07:12.778134 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:12.778150 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:12.778298 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:12.778838 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:12.778843 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:12.779420 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:12.779918 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:12.779937 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:12.781069 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:12.781073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:12.782343 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:12.782360 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:12.783304 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:12.783308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:12.783436 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:12.784060 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:12.784077 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:12.785266 (+ 1189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:12.788836 (+ 3570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:12.788842 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:12.788846 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:12.789343 (+ 497us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:12.789718 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:12.789723 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:12.789727 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:12.789873 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:12.790575 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:12.790580 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:12.790782 (+ 202us) client_negotiation.cc:769] Sending connection context
0809 15:07:12.790940 (+ 158us) client_negotiation.cc:240] Negotiation successful
0809 15:07:12.791258 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":332,"threads_started":1}
I20260809 15:07:12.791728 26943 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.194@KRBTEST.COM'} at 127.24.183.194:53577
WARNING: no policy specified for kudu/127.24.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab.
Entry for principal kudu/127.24.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab.
I20260809 15:07:12.826650 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.195:0
--local_ip_for_outbound_sockets=127.24.183.195
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:42375
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:33567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:13.160993 27468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:13.161259 27468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:13.161391 27468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:13.168430 27468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:13.168542 27468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:13.168664 27468 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 15:07:13.168776 27468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.195
I20260809 15:07:13.178195 27468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:33567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.183.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.183.254:42375
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.195
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:13.179473 27468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:13.180930 27468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:13.193663 27474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:13.193666 27475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:13.193711 27478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:13.194523 27468 server_base.cc:1034] running on GCE node
I20260809 15:07:13.195106 27468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:13.196002 27468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:13.197223 27468 hybrid_clock.cc:648] HybridClock initialized: now 1786288033197153 us; error 47 us; skew 500 ppm
I20260809 15:07:13.197492 27468 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:13 dist-test-slave-nj21 krb5kdc[26887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288033, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:13.200467 27468 init.cc:377] Logged in from keytab as kudu/127.24.183.195@KRBTEST.COM (short username kudu)
I20260809 15:07:13.201871 27468 webserver.cc:460] Webserver started at http://127.24.183.195:41705/ using document root <none> and password file <none>
I20260809 15:07:13.202517 27468 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:13.202615 27468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:13.202931 27468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:13.205768 27468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-2/data/instance:
uuid: "fe6edff75377406780d04ee31436ddce"
format_stamp: "Formatted at 2026-08-09 15:07:13 on dist-test-slave-nj21"
server_key: "f54c3bb93d6777f91c4c200631c2257d"
server_key_iv: "3c54b1febf4bc8d8e5a76bc52ea03e8c"
server_key_version: "encryptionkey@0"
I20260809 15:07:13.206545 27468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance:
uuid: "fe6edff75377406780d04ee31436ddce"
format_stamp: "Formatted at 2026-08-09 15:07:13 on dist-test-slave-nj21"
server_key: "f54c3bb93d6777f91c4c200631c2257d"
server_key_iv: "3c54b1febf4bc8d8e5a76bc52ea03e8c"
server_key_version: "encryptionkey@0"
I20260809 15:07:13.211531 27468 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 15:07:13.215576 27486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:13.216816 27468 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:07:13.216979 27468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-2/wal
uuid: "fe6edff75377406780d04ee31436ddce"
format_stamp: "Formatted at 2026-08-09 15:07:13 on dist-test-slave-nj21"
server_key: "f54c3bb93d6777f91c4c200631c2257d"
server_key_iv: "3c54b1febf4bc8d8e5a76bc52ea03e8c"
server_key_version: "encryptionkey@0"
I20260809 15:07:13.217168 27468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:13.234158 27468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:13.238039 27468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:13.238296 27468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:13.241005 27468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:07:13.241108 27468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:13.241238 27468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:07:13.241324 27468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 15:07:13 dist-test-slave-nj21 krb5kdc[26887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288033, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:13.263620 27083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:13.241672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:43521 (local address 127.24.183.254:42375)
0809 15:07:13.241876 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:13.241883 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:13.243435 (+ 1552us) server_negotiation.cc:407] Connection header received
0809 15:07:13.245600 (+ 2165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:13.245607 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:13.245696 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:13.245785 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:13.247795 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.248820 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.251274 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.251845 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.251985 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:13.257078 (+ 5093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:13.257086 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:13.257091 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:13.257136 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:13.260331 (+ 3195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:13.261233 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:13.261237 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:13.261239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:13.261394 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:13.261868 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:13.261873 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:13.261875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:13.262181 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:13.262294 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:13.263187 (+ 893us) server_negotiation.cc:299] Negotiation successful
0809 15:07:13.263356 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 15:07:13.265369 27498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:13.242296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:42375 (local address 127.24.183.195:43521)
0809 15:07:13.243276 (+ 980us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:13.243318 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:13.245320 (+ 2002us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:13.246040 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:13.246047 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:13.246728 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:13.247582 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:13.247619 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.249007 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.249010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:13.251132 (+ 2122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:13.251147 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.252286 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.252297 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:13.252575 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:13.253332 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:13.253363 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:13.256759 (+ 3396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:13.260463 (+ 3704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:13.260474 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:13.260493 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:13.261105 (+ 612us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:13.261487 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:13.261492 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:13.261496 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:13.261751 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:13.262307 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:13.262317 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:13.262777 (+ 460us) client_negotiation.cc:769] Sending connection context
0809 15:07:13.263081 (+ 304us) client_negotiation.cc:240] Negotiation successful
0809 15:07:13.263645 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":727,"thread_start_us":437,"threads_started":1}
I20260809 15:07:13.285033 27083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:13.270746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:56167 (local address 127.24.183.254:42375)
0809 15:07:13.270926 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:13.270931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:13.271545 (+ 614us) server_negotiation.cc:407] Connection header received
0809 15:07:13.271743 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:13.271747 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:13.271821 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:13.271897 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:13.273106 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.273782 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.275759 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.276307 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.276435 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:13.278641 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:13.278647 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:13.278652 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:13.278691 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:13.282179 (+ 3488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:13.283043 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:13.283049 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:13.283052 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:13.283210 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:13.283652 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:13.283657 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:13.283661 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:13.284024 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:13.284187 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:13.284626 (+ 439us) server_negotiation.cc:299] Negotiation successful
0809 15:07:13.284787 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 15:07:13.285353 27585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:13.270743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:42375 (local address 127.24.183.195:56167)
0809 15:07:13.271440 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:13.271460 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:13.271602 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:13.271940 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:13.271945 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:13.272579 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:13.272978 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:13.272991 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.273898 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.273904 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:13.275591 (+ 1687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:13.275608 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.276516 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.276523 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:13.276724 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:13.277308 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:13.277339 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:13.278434 (+ 1095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:13.282319 (+ 3885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:13.282325 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:13.282337 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:13.282891 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:13.283367 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:13.283373 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:13.283377 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:13.283547 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:13.284150 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:13.284156 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:13.284378 (+ 222us) client_negotiation.cc:769] Sending connection context
0809 15:07:13.284541 (+ 163us) client_negotiation.cc:240] Negotiation successful
0809 15:07:13.284885 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":369,"threads_started":1}
I20260809 15:07:13.288471 27252 heartbeater.cc:503] Master 127.24.183.254:42375 was elected leader, sending a full tablet report...
I20260809 15:07:13.303767 27468 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.195:45523
I20260809 15:07:13.303792 27666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.195:45523 every 8 connection(s)
I20260809 15:07:13.304988 27468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
I20260809 15:07:13.306897 27667 heartbeater.cc:347] Connected to a master server at 127.24.183.254:42375
I20260809 15:07:13.307094 27667 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:13.307732 27667 heartbeater.cc:511] Master 127.24.183.254:42375 requested a full tablet report, sending...
I20260809 15:07:13.309024 26943 ts_manager.cc:194] Registered new tserver with Master: fe6edff75377406780d04ee31436ddce (127.24.183.195:45523)
I20260809 15:07:13.309850 26943 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.195@KRBTEST.COM'} at 127.24.183.195:43521
I20260809 15:07:13.313192 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 27468
I20260809 15:07:13.313401 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance
I20260809 15:07:13.313899 25311 external_mini_cluster.cc:1428] Setting key df661193174d5dd336660a2c1be80f57
I20260809 15:07:13.316884 25311 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 15:07:13 dist-test-slave-nj21 krb5kdc[26887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288033, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 15:07:13 dist-test-slave-nj21 krb5kdc[26887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288033, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:13.364022 27083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:13.350884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59836 (local address 127.24.183.254:42375)
0809 15:07:13.351101 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:13.351108 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:13.351311 (+ 203us) server_negotiation.cc:407] Connection header received
0809 15:07:13.351459 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:13.351463 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:13.351530 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:13.351613 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:13.352947 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.353832 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.355066 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.355479 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.355570 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:13.359177 (+ 3607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:13.359182 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:13.359185 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:13.359216 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:13.361594 (+ 2378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:13.362257 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:13.362262 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:13.362264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:13.362442 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:13.362789 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:13.362791 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:13.362793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:13.363102 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:13.363281 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:13.363511 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 15:07:13.363710 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 15:07:13.368012 26943 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59836:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:07:13.374879 26943 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:07:13.400653 27690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:13.388953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52992 (local address 127.24.183.194:41627)
0809 15:07:13.392559 (+ 3606us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:13.392569 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:13.392601 (+ 32us) server_negotiation.cc:407] Connection header received
0809 15:07:13.392708 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:13.392727 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:13.392973 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:13.393171 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:07:13.394022 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.395216 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.398267 (+ 3051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.399551 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.399742 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:13.399934 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:13.400188 (+ 254us) server_negotiation.cc:299] Negotiation successful
0809 15:07:13.400355 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3462,"thread_start_us":3329,"threads_started":1}
I20260809 15:07:13.400764 27686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:13.388435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:41627 (local address 127.0.0.1:52992)
0809 15:07:13.391965 (+ 3530us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:13.391981 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:13.392080 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:13.393232 (+ 1152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:13.393240 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:13.393276 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:07:13.393675 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:13.393688 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.395762 (+ 2074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.395766 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:13.398100 (+ 2334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:13.398117 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.399765 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.399771 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:13.399968 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:13.400015 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 15:07:13.400123 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 15:07:13.400321 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3425,"mutex_wait_us":65,"thread_start_us":2622,"threads_started":1}
I20260809 15:07:13.401366 27689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:13.386744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:45523 (local address 127.0.0.1:52412)
0809 15:07:13.391350 (+ 4606us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:13.391369 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:13.391485 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:13.393627 (+ 2142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:13.393632 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:13.393659 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:07:13.394140 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:13.394153 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.396322 (+ 2169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.396338 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:13.398762 (+ 2424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:13.398777 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.400430 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.400436 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:13.400603 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:13.400640 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 15:07:13.400713 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 15:07:13.400848 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4450,"mutex_wait_us":330,"thread_start_us":4976,"threads_started":1}
I20260809 15:07:13.401809 27685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:13.386574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:40361 (local address 127.0.0.1:58058)
0809 15:07:13.388919 (+ 2345us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:13.388964 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:13.389138 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:13.395355 (+ 6217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:13.395358 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:13.395379 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:07:13.395652 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:13.395659 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.397103 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.397109 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:13.399168 (+ 2059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:13.399183 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.401020 (+ 1837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.401027 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:13.401208 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:13.401242 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 15:07:13.401314 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 15:07:13.401450 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2094,"mutex_wait_us":2,"thread_start_us":2745,"threads_started":1}
I20260809 15:07:13.402707 27687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:13.386871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58058 (local address 127.24.183.193:40361)
0809 15:07:13.394811 (+ 7940us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:13.394819 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:13.394847 (+ 28us) server_negotiation.cc:407] Connection header received
0809 15:07:13.394940 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:13.394948 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:13.395156 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:13.395341 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:07:13.395883 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.396964 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.399355 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.400796 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.402057 (+ 1261us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:13.402212 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:13.402268 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 15:07:13.402432 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7759,"thread_start_us":7622,"threads_started":1}
I20260809 15:07:13.402792 27688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:13.387459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52412 (local address 127.24.183.195:45523)
0809 15:07:13.393045 (+ 5586us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:13.393052 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:13.393082 (+ 30us) server_negotiation.cc:407] Connection header received
0809 15:07:13.393181 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:13.393188 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:13.393406 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:13.394346 (+ 940us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:07:13.394698 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.396172 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.398913 (+ 2741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.400197 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.401867 (+ 1670us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:13.402045 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:13.402120 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 15:07:13.402267 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5423,"thread_start_us":5218,"threads_started":1}
I20260809 15:07:13.404830 27366 tablet_service.cc:1467] Processing CreateTablet for tablet cbd821ab0e724a00a7e06624c91e05d1 (DEFAULT_TABLE table=test-table [id=6c26dab3d42543069793ac9630ddc7b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:07:13.405560 27561 tablet_service.cc:1467] Processing CreateTablet for tablet cbd821ab0e724a00a7e06624c91e05d1 (DEFAULT_TABLE table=test-table [id=6c26dab3d42543069793ac9630ddc7b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:07:13.406375 27366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbd821ab0e724a00a7e06624c91e05d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:13.406077 27150 tablet_service.cc:1467] Processing CreateTablet for tablet cbd821ab0e724a00a7e06624c91e05d1 (DEFAULT_TABLE table=test-table [id=6c26dab3d42543069793ac9630ddc7b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:07:13.406621 27561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbd821ab0e724a00a7e06624c91e05d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:13.407368 27150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbd821ab0e724a00a7e06624c91e05d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:13.417940 27691 tablet_bootstrap.cc:492] T cbd821ab0e724a00a7e06624c91e05d1 P fe6edff75377406780d04ee31436ddce: Bootstrap starting.
I20260809 15:07:13.420395 27695 tablet_bootstrap.cc:492] T cbd821ab0e724a00a7e06624c91e05d1 P ba2c864293ad499f8e13bee4f688cbd1: Bootstrap starting.
I20260809 15:07:13.421514 27691 tablet_bootstrap.cc:654] T cbd821ab0e724a00a7e06624c91e05d1 P fe6edff75377406780d04ee31436ddce: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:13.421788 27696 tablet_bootstrap.cc:492] T cbd821ab0e724a00a7e06624c91e05d1 P be36dd1898cd4b008b8295d3923410bd: Bootstrap starting.
I20260809 15:07:13.422904 27691 log.cc:824] T cbd821ab0e724a00a7e06624c91e05d1 P fe6edff75377406780d04ee31436ddce: Log is configured to *not* fsync() on all Append() calls
I20260809 15:07:13.424376 27695 tablet_bootstrap.cc:654] T cbd821ab0e724a00a7e06624c91e05d1 P ba2c864293ad499f8e13bee4f688cbd1: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:13.424945 27696 tablet_bootstrap.cc:654] T cbd821ab0e724a00a7e06624c91e05d1 P be36dd1898cd4b008b8295d3923410bd: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:13.425640 27695 log.cc:824] T cbd821ab0e724a00a7e06624c91e05d1 P ba2c864293ad499f8e13bee4f688cbd1: Log is configured to *not* fsync() on all Append() calls
I20260809 15:07:13.425891 27696 log.cc:824] T cbd821ab0e724a00a7e06624c91e05d1 P be36dd1898cd4b008b8295d3923410bd: Log is configured to *not* fsync() on all Append() calls
I20260809 15:07:13.426554 27691 tablet_bootstrap.cc:492] T cbd821ab0e724a00a7e06624c91e05d1 P fe6edff75377406780d04ee31436ddce: No bootstrap required, opened a new log
I20260809 15:07:13.426883 27691 ts_tablet_manager.cc:1397] T cbd821ab0e724a00a7e06624c91e05d1 P fe6edff75377406780d04ee31436ddce: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260809 15:07:13.428439 27695 tablet_bootstrap.cc:492] T cbd821ab0e724a00a7e06624c91e05d1 P ba2c864293ad499f8e13bee4f688cbd1: No bootstrap required, opened a new log
I20260809 15:07:13.428704 27695 ts_tablet_manager.cc:1397] T cbd821ab0e724a00a7e06624c91e05d1 P ba2c864293ad499f8e13bee4f688cbd1: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 15:07:13.428855 27696 tablet_bootstrap.cc:492] T cbd821ab0e724a00a7e06624c91e05d1 P be36dd1898cd4b008b8295d3923410bd: No bootstrap required, opened a new log
I20260809 15:07:13.429111 27696 ts_tablet_manager.cc:1397] T cbd821ab0e724a00a7e06624c91e05d1 P be36dd1898cd4b008b8295d3923410bd: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.005s
I20260809 15:07:13.433845 27691 raft_consensus.cc:348] T cbd821ab0e724a00a7e06624c91e05d1 P fe6edff75377406780d04ee31436ddce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } }
I20260809 15:07:13.434172 27691 raft_consensus.cc:374] T cbd821ab0e724a00a7e06624c91e05d1 P fe6edff75377406780d04ee31436ddce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:13.434273 27691 raft_consensus.cc:729] T cbd821ab0e724a00a7e06624c91e05d1 P fe6edff75377406780d04ee31436ddce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe6edff75377406780d04ee31436ddce, State: Initialized, Role: FOLLOWER
I20260809 15:07:13.435194 27691 consensus_queue.cc:260] T cbd821ab0e724a00a7e06624c91e05d1 P fe6edff75377406780d04ee31436ddce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } }
I20260809 15:07:13.436378 27667 heartbeater.cc:503] Master 127.24.183.254:42375 was elected leader, sending a full tablet report...
I20260809 15:07:13.437136 27691 ts_tablet_manager.cc:1428] T cbd821ab0e724a00a7e06624c91e05d1 P fe6edff75377406780d04ee31436ddce: Time spent starting tablet: real 0.010s user 0.005s sys 0.004s
I20260809 15:07:13.438956 27695 raft_consensus.cc:348] T cbd821ab0e724a00a7e06624c91e05d1 P ba2c864293ad499f8e13bee4f688cbd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } }
I20260809 15:07:13.439047 27696 raft_consensus.cc:348] T cbd821ab0e724a00a7e06624c91e05d1 P be36dd1898cd4b008b8295d3923410bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } }
I20260809 15:07:13.439397 27695 raft_consensus.cc:374] T cbd821ab0e724a00a7e06624c91e05d1 P ba2c864293ad499f8e13bee4f688cbd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:13.439397 27696 raft_consensus.cc:374] T cbd821ab0e724a00a7e06624c91e05d1 P be36dd1898cd4b008b8295d3923410bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:13.439504 27695 raft_consensus.cc:729] T cbd821ab0e724a00a7e06624c91e05d1 P ba2c864293ad499f8e13bee4f688cbd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2c864293ad499f8e13bee4f688cbd1, State: Initialized, Role: FOLLOWER
I20260809 15:07:13.439500 27696 raft_consensus.cc:729] T cbd821ab0e724a00a7e06624c91e05d1 P be36dd1898cd4b008b8295d3923410bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be36dd1898cd4b008b8295d3923410bd, State: Initialized, Role: FOLLOWER
I20260809 15:07:13.440106 27696 consensus_queue.cc:260] T cbd821ab0e724a00a7e06624c91e05d1 P be36dd1898cd4b008b8295d3923410bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } }
I20260809 15:07:13.440160 27695 consensus_queue.cc:260] T cbd821ab0e724a00a7e06624c91e05d1 P ba2c864293ad499f8e13bee4f688cbd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } }
I20260809 15:07:13.441790 27696 ts_tablet_manager.cc:1428] T cbd821ab0e724a00a7e06624c91e05d1 P be36dd1898cd4b008b8295d3923410bd: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260809 15:07:13.442577 27463 heartbeater.cc:503] Master 127.24.183.254:42375 was elected leader, sending a full tablet report...
I20260809 15:07:13.443120 27695 ts_tablet_manager.cc:1428] T cbd821ab0e724a00a7e06624c91e05d1 P ba2c864293ad499f8e13bee4f688cbd1: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260809 15:07:13.460238 27700 raft_consensus.cc:482] T cbd821ab0e724a00a7e06624c91e05d1 P fe6edff75377406780d04ee31436ddce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:07:13.460505 27700 raft_consensus.cc:504] T cbd821ab0e724a00a7e06624c91e05d1 P fe6edff75377406780d04ee31436ddce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } }
I20260809 15:07:13.462024 27700 leader_election.cc:288] T cbd821ab0e724a00a7e06624c91e05d1 P fe6edff75377406780d04ee31436ddce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be36dd1898cd4b008b8295d3923410bd (127.24.183.193:40361), ba2c864293ad499f8e13bee4f688cbd1 (127.24.183.194:41627)
I20260809 15:07:13.468319 27690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:13.462459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:36217 (local address 127.24.183.194:41627)
0809 15:07:13.462636 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:13.462641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:13.462660 (+ 19us) server_negotiation.cc:407] Connection header received
0809 15:07:13.462741 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:13.462746 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:13.462818 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:13.462905 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:07:13.463495 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.464532 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.466355 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.467410 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.467526 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:13.467579 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:13.467962 (+ 383us) server_negotiation.cc:299] Negotiation successful
0809 15:07:13.468076 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 15:07:13.468416 27498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:13.462266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:41627 (local address 127.24.183.195:36217)
0809 15:07:13.462490 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:13.462510 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:13.462632 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:13.463077 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:13.463082 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:13.463109 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:07:13.463322 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:13.463334 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.464652 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.464658 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:13.466204 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:13.466217 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.467545 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.467573 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:13.467806 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:13.467844 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 15:07:13.467939 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 15:07:13.468097 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 15:07:13.469080 27703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:13.462346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:40361 (local address 127.24.183.195:35985)
0809 15:07:13.463195 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:13.463221 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:13.463342 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:13.463768 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:13.463773 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:13.463799 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:07:13.464135 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:13.464147 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.465367 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.465372 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:13.466996 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:13.467011 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.468451 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.468457 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:13.468620 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:13.468652 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 15:07:13.468749 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 15:07:13.468858 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"thread_start_us":524,"threads_started":1}
I20260809 15:07:13.469146 27687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:13.462548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:35985 (local address 127.24.183.193:40361)
0809 15:07:13.462748 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:13.462754 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:13.463301 (+ 547us) server_negotiation.cc:407] Connection header received
0809 15:07:13.463466 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:13.463471 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:13.463558 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:13.463648 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:07:13.464313 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.465240 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.467138 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.468200 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.468351 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:13.468416 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:13.468794 (+ 378us) server_negotiation.cc:299] Negotiation successful
0809 15:07:13.468930 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 15:07:13.469585 27397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbd821ab0e724a00a7e06624c91e05d1" candidate_uuid: "fe6edff75377406780d04ee31436ddce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2c864293ad499f8e13bee4f688cbd1" is_pre_election: true
I20260809 15:07:13.470084 27397 raft_consensus.cc:2454] T cbd821ab0e724a00a7e06624c91e05d1 P ba2c864293ad499f8e13bee4f688cbd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe6edff75377406780d04ee31436ddce in term 0.
I20260809 15:07:13.470079 27183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbd821ab0e724a00a7e06624c91e05d1" candidate_uuid: "fe6edff75377406780d04ee31436ddce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be36dd1898cd4b008b8295d3923410bd" is_pre_election: true
I20260809 15:07:13.470543 27183 raft_consensus.cc:2454] T cbd821ab0e724a00a7e06624c91e05d1 P be36dd1898cd4b008b8295d3923410bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe6edff75377406780d04ee31436ddce in term 0.
I20260809 15:07:13.470672 27491 leader_election.cc:302] T cbd821ab0e724a00a7e06624c91e05d1 P fe6edff75377406780d04ee31436ddce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2c864293ad499f8e13bee4f688cbd1, fe6edff75377406780d04ee31436ddce; no voters:
I20260809 15:07:13.471042 27700 raft_consensus.cc:2781] T cbd821ab0e724a00a7e06624c91e05d1 P fe6edff75377406780d04ee31436ddce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:07:13.471179 27700 raft_consensus.cc:482] T cbd821ab0e724a00a7e06624c91e05d1 P fe6edff75377406780d04ee31436ddce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:07:13.471276 27700 raft_consensus.cc:3037] T cbd821ab0e724a00a7e06624c91e05d1 P fe6edff75377406780d04ee31436ddce [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:13.473512 27700 raft_consensus.cc:504] T cbd821ab0e724a00a7e06624c91e05d1 P fe6edff75377406780d04ee31436ddce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } }
I20260809 15:07:13.474059 27700 leader_election.cc:288] T cbd821ab0e724a00a7e06624c91e05d1 P fe6edff75377406780d04ee31436ddce [CANDIDATE]: Term 1 election: Requested vote from peers be36dd1898cd4b008b8295d3923410bd (127.24.183.193:40361), ba2c864293ad499f8e13bee4f688cbd1 (127.24.183.194:41627)
I20260809 15:07:13.474475 27183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbd821ab0e724a00a7e06624c91e05d1" candidate_uuid: "fe6edff75377406780d04ee31436ddce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be36dd1898cd4b008b8295d3923410bd"
I20260809 15:07:13.474499 27397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbd821ab0e724a00a7e06624c91e05d1" candidate_uuid: "fe6edff75377406780d04ee31436ddce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2c864293ad499f8e13bee4f688cbd1"
I20260809 15:07:13.474642 27183 raft_consensus.cc:3037] T cbd821ab0e724a00a7e06624c91e05d1 P be36dd1898cd4b008b8295d3923410bd [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:13.474649 27397 raft_consensus.cc:3037] T cbd821ab0e724a00a7e06624c91e05d1 P ba2c864293ad499f8e13bee4f688cbd1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:13.476845 27397 raft_consensus.cc:2454] T cbd821ab0e724a00a7e06624c91e05d1 P ba2c864293ad499f8e13bee4f688cbd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe6edff75377406780d04ee31436ddce in term 1.
I20260809 15:07:13.476990 27183 raft_consensus.cc:2454] T cbd821ab0e724a00a7e06624c91e05d1 P be36dd1898cd4b008b8295d3923410bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe6edff75377406780d04ee31436ddce in term 1.
I20260809 15:07:13.477212 27491 leader_election.cc:302] T cbd821ab0e724a00a7e06624c91e05d1 P fe6edff75377406780d04ee31436ddce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2c864293ad499f8e13bee4f688cbd1, fe6edff75377406780d04ee31436ddce; no voters:
I20260809 15:07:13.477478 27700 raft_consensus.cc:2781] T cbd821ab0e724a00a7e06624c91e05d1 P fe6edff75377406780d04ee31436ddce [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:07:13.477939 27700 raft_consensus.cc:686] T cbd821ab0e724a00a7e06624c91e05d1 P fe6edff75377406780d04ee31436ddce [term 1 LEADER]: Becoming Leader. State: Replica: fe6edff75377406780d04ee31436ddce, State: Running, Role: LEADER
I20260809 15:07:13.478433 27700 consensus_queue.cc:237] T cbd821ab0e724a00a7e06624c91e05d1 P fe6edff75377406780d04ee31436ddce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } }
I20260809 15:07:13.483091 26942 catalog_manager.cc:5495] T cbd821ab0e724a00a7e06624c91e05d1 P fe6edff75377406780d04ee31436ddce reported cstate change: term changed from 0 to 1, leader changed from <none> to fe6edff75377406780d04ee31436ddce (127.24.183.195). New cstate: current_term: 1 leader_uuid: "fe6edff75377406780d04ee31436ddce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } health_report { overall_health: UNKNOWN } } }
I20260809 15:07:13.526094 27688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:13.520157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52428 (local address 127.24.183.195:45523)
0809 15:07:13.520374 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:13.520382 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:13.520519 (+ 137us) server_negotiation.cc:407] Connection header received
0809 15:07:13.520871 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:13.520875 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:13.520975 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:13.521116 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:07:13.521544 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.522311 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.523773 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.524233 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.524362 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:13.524892 (+ 530us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:07:13.525420 (+ 528us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:07:13.525562 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:13.525720 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 15:07:13.525815 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260809 15:07:13.532114 27254 tablet.cc:2369] T cbd821ab0e724a00a7e06624c91e05d1 P be36dd1898cd4b008b8295d3923410bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:07:13.534365 27183 raft_consensus.cc:1261] T cbd821ab0e724a00a7e06624c91e05d1 P be36dd1898cd4b008b8295d3923410bd [term 1 FOLLOWER]: Refusing update from remote peer fe6edff75377406780d04ee31436ddce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:07:13.534457 27397 raft_consensus.cc:1261] T cbd821ab0e724a00a7e06624c91e05d1 P ba2c864293ad499f8e13bee4f688cbd1 [term 1 FOLLOWER]: Refusing update from remote peer fe6edff75377406780d04ee31436ddce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:07:13.535271 27705 consensus_queue.cc:1035] T cbd821ab0e724a00a7e06624c91e05d1 P fe6edff75377406780d04ee31436ddce [LEADER]: Connected to new peer: Peer: permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:07:13.535620 27700 consensus_queue.cc:1035] T cbd821ab0e724a00a7e06624c91e05d1 P fe6edff75377406780d04ee31436ddce [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 15:07:13.535986 27464 tablet.cc:2369] T cbd821ab0e724a00a7e06624c91e05d1 P ba2c864293ad499f8e13bee4f688cbd1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:07:13.546725 27709 mvcc.cc:205] Tried to move back new op lower bound from 7316635785346056192 to 7316635785131683840. Current Snapshot: MvccSnapshot[applied={T|T < 7316635785346056192}]
I20260809 15:07:13.569993 26943 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:07:13.570323 26943 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:07:13.574177 26943 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a7c251fa7fa546fc86a1d5e92878800e: Sending DeleteTablet for 3 replicas of tablet cbd821ab0e724a00a7e06624c91e05d1
I20260809 15:07:13.575304 27561 tablet_service.cc:1514] Processing DeleteTablet for tablet cbd821ab0e724a00a7e06624c91e05d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:07:13 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:52412
I20260809 15:07:13.575321 27150 tablet_service.cc:1514] Processing DeleteTablet for tablet cbd821ab0e724a00a7e06624c91e05d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:07:13 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:58058
I20260809 15:07:13.575598 27366 tablet_service.cc:1514] Processing DeleteTablet for tablet cbd821ab0e724a00a7e06624c91e05d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:07:13 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:52992
I20260809 15:07:13.578433 26942 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59836:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:07:13.579233 26942 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:07:13.587438 27365 tablet_service.cc:1467] Processing CreateTablet for tablet 6016eada77a14f918a2c29c64593edf7 (DEFAULT_TABLE table=test-table [id=94cb9ec198534d64a4d2ae7e2260e433]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:07:13.587538 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 6016eada77a14f918a2c29c64593edf7 (DEFAULT_TABLE table=test-table [id=94cb9ec198534d64a4d2ae7e2260e433]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:07:13.588095 27365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6016eada77a14f918a2c29c64593edf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:13.588254 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6016eada77a14f918a2c29c64593edf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:13.592877 27149 tablet_service.cc:1467] Processing CreateTablet for tablet 6016eada77a14f918a2c29c64593edf7 (DEFAULT_TABLE table=test-table [id=94cb9ec198534d64a4d2ae7e2260e433]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:07:13.593465 27149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6016eada77a14f918a2c29c64593edf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:13.595638 27691 tablet_bootstrap.cc:492] T 6016eada77a14f918a2c29c64593edf7 P fe6edff75377406780d04ee31436ddce: Bootstrap starting.
I20260809 15:07:13.596350 27695 tablet_bootstrap.cc:492] T 6016eada77a14f918a2c29c64593edf7 P ba2c864293ad499f8e13bee4f688cbd1: Bootstrap starting.
I20260809 15:07:13.597450 27691 tablet_bootstrap.cc:654] T 6016eada77a14f918a2c29c64593edf7 P fe6edff75377406780d04ee31436ddce: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:13.598067 27695 tablet_bootstrap.cc:654] T 6016eada77a14f918a2c29c64593edf7 P ba2c864293ad499f8e13bee4f688cbd1: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:13.601621 27696 tablet_bootstrap.cc:492] T 6016eada77a14f918a2c29c64593edf7 P be36dd1898cd4b008b8295d3923410bd: Bootstrap starting.
I20260809 15:07:13.603240 27696 tablet_bootstrap.cc:654] T 6016eada77a14f918a2c29c64593edf7 P be36dd1898cd4b008b8295d3923410bd: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:13.603274 27729 tablet_replica.cc:330] T cbd821ab0e724a00a7e06624c91e05d1 P ba2c864293ad499f8e13bee4f688cbd1: stopping tablet replica
I20260809 15:07:13.605975 27729 raft_consensus.cc:2229] T cbd821ab0e724a00a7e06624c91e05d1 P ba2c864293ad499f8e13bee4f688cbd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:07:13.607100 27691 tablet_bootstrap.cc:492] T 6016eada77a14f918a2c29c64593edf7 P fe6edff75377406780d04ee31436ddce: No bootstrap required, opened a new log
I20260809 15:07:13.607318 27691 ts_tablet_manager.cc:1397] T 6016eada77a14f918a2c29c64593edf7 P fe6edff75377406780d04ee31436ddce: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.003s
I20260809 15:07:13.608042 27729 raft_consensus.cc:2258] T cbd821ab0e724a00a7e06624c91e05d1 P ba2c864293ad499f8e13bee4f688cbd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:07:13.608320 27691 raft_consensus.cc:348] T 6016eada77a14f918a2c29c64593edf7 P fe6edff75377406780d04ee31436ddce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } }
I20260809 15:07:13.608646 27691 raft_consensus.cc:374] T 6016eada77a14f918a2c29c64593edf7 P fe6edff75377406780d04ee31436ddce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:13.608796 27691 raft_consensus.cc:729] T 6016eada77a14f918a2c29c64593edf7 P fe6edff75377406780d04ee31436ddce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe6edff75377406780d04ee31436ddce, State: Initialized, Role: FOLLOWER
I20260809 15:07:13.609205 27691 consensus_queue.cc:260] T 6016eada77a14f918a2c29c64593edf7 P fe6edff75377406780d04ee31436ddce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } }
I20260809 15:07:13.609879 27691 ts_tablet_manager.cc:1428] T 6016eada77a14f918a2c29c64593edf7 P fe6edff75377406780d04ee31436ddce: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 15:07:13.610549 27721 tablet_replica.cc:330] T cbd821ab0e724a00a7e06624c91e05d1 P fe6edff75377406780d04ee31436ddce: stopping tablet replica
I20260809 15:07:13.611024 27721 raft_consensus.cc:2229] T cbd821ab0e724a00a7e06624c91e05d1 P fe6edff75377406780d04ee31436ddce [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:07:13.611759 27695 tablet_bootstrap.cc:492] T 6016eada77a14f918a2c29c64593edf7 P ba2c864293ad499f8e13bee4f688cbd1: No bootstrap required, opened a new log
I20260809 15:07:13.611819 27721 raft_consensus.cc:2258] T cbd821ab0e724a00a7e06624c91e05d1 P fe6edff75377406780d04ee31436ddce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:07:13.611967 27695 ts_tablet_manager.cc:1397] T 6016eada77a14f918a2c29c64593edf7 P ba2c864293ad499f8e13bee4f688cbd1: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.008s
I20260809 15:07:13.612794 27695 raft_consensus.cc:348] T 6016eada77a14f918a2c29c64593edf7 P ba2c864293ad499f8e13bee4f688cbd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } }
I20260809 15:07:13.613073 27695 raft_consensus.cc:374] T 6016eada77a14f918a2c29c64593edf7 P ba2c864293ad499f8e13bee4f688cbd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:13.613214 27695 raft_consensus.cc:729] T 6016eada77a14f918a2c29c64593edf7 P ba2c864293ad499f8e13bee4f688cbd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2c864293ad499f8e13bee4f688cbd1, State: Initialized, Role: FOLLOWER
I20260809 15:07:13.613540 27695 consensus_queue.cc:260] T 6016eada77a14f918a2c29c64593edf7 P ba2c864293ad499f8e13bee4f688cbd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } }
I20260809 15:07:13.614853 27721 ts_tablet_manager.cc:1905] T cbd821ab0e724a00a7e06624c91e05d1 P fe6edff75377406780d04ee31436ddce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:07:13.615269 27729 ts_tablet_manager.cc:1905] T cbd821ab0e724a00a7e06624c91e05d1 P ba2c864293ad499f8e13bee4f688cbd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:07:13.617069 27695 ts_tablet_manager.cc:1428] T 6016eada77a14f918a2c29c64593edf7 P ba2c864293ad499f8e13bee4f688cbd1: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260809 15:07:13.621362 27728 tablet_replica.cc:330] T cbd821ab0e724a00a7e06624c91e05d1 P be36dd1898cd4b008b8295d3923410bd: stopping tablet replica
I20260809 15:07:13.621863 27728 raft_consensus.cc:2229] T cbd821ab0e724a00a7e06624c91e05d1 P be36dd1898cd4b008b8295d3923410bd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:07:13.622032 27721 ts_tablet_manager.cc:1918] T cbd821ab0e724a00a7e06624c91e05d1 P fe6edff75377406780d04ee31436ddce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:07:13.622217 27721 log.cc:1196] T cbd821ab0e724a00a7e06624c91e05d1 P fe6edff75377406780d04ee31436ddce: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-2/wal/wals/cbd821ab0e724a00a7e06624c91e05d1
I20260809 15:07:13.622467 27728 raft_consensus.cc:2258] T cbd821ab0e724a00a7e06624c91e05d1 P be36dd1898cd4b008b8295d3923410bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:07:13.622741 27729 ts_tablet_manager.cc:1918] T cbd821ab0e724a00a7e06624c91e05d1 P ba2c864293ad499f8e13bee4f688cbd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:07:13.622907 27729 log.cc:1196] T cbd821ab0e724a00a7e06624c91e05d1 P ba2c864293ad499f8e13bee4f688cbd1: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-1/wal/wals/cbd821ab0e724a00a7e06624c91e05d1
I20260809 15:07:13.623083 27721 ts_tablet_manager.cc:1939] T cbd821ab0e724a00a7e06624c91e05d1 P fe6edff75377406780d04ee31436ddce: Deleting consensus metadata
I20260809 15:07:13.623430 27729 ts_tablet_manager.cc:1939] T cbd821ab0e724a00a7e06624c91e05d1 P ba2c864293ad499f8e13bee4f688cbd1: Deleting consensus metadata
I20260809 15:07:13.623539 27696 tablet_bootstrap.cc:492] T 6016eada77a14f918a2c29c64593edf7 P be36dd1898cd4b008b8295d3923410bd: No bootstrap required, opened a new log
I20260809 15:07:13.623692 27696 ts_tablet_manager.cc:1397] T 6016eada77a14f918a2c29c64593edf7 P be36dd1898cd4b008b8295d3923410bd: Time spent bootstrapping tablet: real 0.022s user 0.007s sys 0.000s
I20260809 15:07:13.624480 27696 raft_consensus.cc:348] T 6016eada77a14f918a2c29c64593edf7 P be36dd1898cd4b008b8295d3923410bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } }
I20260809 15:07:13.624699 27696 raft_consensus.cc:374] T 6016eada77a14f918a2c29c64593edf7 P be36dd1898cd4b008b8295d3923410bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:13.624799 27696 raft_consensus.cc:729] T 6016eada77a14f918a2c29c64593edf7 P be36dd1898cd4b008b8295d3923410bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be36dd1898cd4b008b8295d3923410bd, State: Initialized, Role: FOLLOWER
I20260809 15:07:13.625026 27696 consensus_queue.cc:260] T 6016eada77a14f918a2c29c64593edf7 P be36dd1898cd4b008b8295d3923410bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } }
I20260809 15:07:13.625228 26925 catalog_manager.cc:4841] TS ba2c864293ad499f8e13bee4f688cbd1 (127.24.183.194:41627): tablet cbd821ab0e724a00a7e06624c91e05d1 (table test-table [id=6c26dab3d42543069793ac9630ddc7b4]) successfully deleted
I20260809 15:07:13.625412 27728 ts_tablet_manager.cc:1905] T cbd821ab0e724a00a7e06624c91e05d1 P be36dd1898cd4b008b8295d3923410bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:07:13.625562 27696 ts_tablet_manager.cc:1428] T 6016eada77a14f918a2c29c64593edf7 P be36dd1898cd4b008b8295d3923410bd: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 15:07:13.626663 26926 catalog_manager.cc:4841] TS fe6edff75377406780d04ee31436ddce (127.24.183.195:45523): tablet cbd821ab0e724a00a7e06624c91e05d1 (table test-table [id=6c26dab3d42543069793ac9630ddc7b4]) successfully deleted
I20260809 15:07:13.633642 27728 ts_tablet_manager.cc:1918] T cbd821ab0e724a00a7e06624c91e05d1 P be36dd1898cd4b008b8295d3923410bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:07:13.633883 27728 log.cc:1196] T cbd821ab0e724a00a7e06624c91e05d1 P be36dd1898cd4b008b8295d3923410bd: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-0/wal/wals/cbd821ab0e724a00a7e06624c91e05d1
I20260809 15:07:13.634187 27701 raft_consensus.cc:482] T 6016eada77a14f918a2c29c64593edf7 P be36dd1898cd4b008b8295d3923410bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:07:13.635017 27728 ts_tablet_manager.cc:1939] T cbd821ab0e724a00a7e06624c91e05d1 P be36dd1898cd4b008b8295d3923410bd: Deleting consensus metadata
I20260809 15:07:13.634492 27701 raft_consensus.cc:504] T 6016eada77a14f918a2c29c64593edf7 P be36dd1898cd4b008b8295d3923410bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } }
I20260809 15:07:13.645226 26923 catalog_manager.cc:4841] TS be36dd1898cd4b008b8295d3923410bd (127.24.183.193:40361): tablet cbd821ab0e724a00a7e06624c91e05d1 (table test-table [id=6c26dab3d42543069793ac9630ddc7b4]) successfully deleted
I20260809 15:07:13.645429 27701 leader_election.cc:288] T 6016eada77a14f918a2c29c64593edf7 P be36dd1898cd4b008b8295d3923410bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba2c864293ad499f8e13bee4f688cbd1 (127.24.183.194:41627), fe6edff75377406780d04ee31436ddce (127.24.183.195:45523)
I20260809 15:07:13.655264 27690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:13.637796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:51349 (local address 127.24.183.194:41627)
0809 15:07:13.637994 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:13.638001 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:13.648960 (+ 10959us) server_negotiation.cc:407] Connection header received
0809 15:07:13.649303 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:13.649310 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:13.649394 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:13.649482 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:07:13.650309 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.651440 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.653313 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.654211 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.654358 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:13.654435 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:13.654824 (+ 389us) server_negotiation.cc:299] Negotiation successful
0809 15:07:13.654976 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 15:07:13.655344 27737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:13.637393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:41627 (local address 127.24.183.193:51349)
0809 15:07:13.648843 (+ 11450us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:13.648864 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:13.649109 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:13.649709 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:13.649714 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:13.649742 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:07:13.650069 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:13.650082 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.651583 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.651588 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:13.653161 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:13.653173 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.654463 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.654470 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:13.654654 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:13.654690 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 15:07:13.654787 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 15:07:13.654917 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11319,"thread_start_us":11955,"threads_started":1}
I20260809 15:07:13.656497 27397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6016eada77a14f918a2c29c64593edf7" candidate_uuid: "be36dd1898cd4b008b8295d3923410bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2c864293ad499f8e13bee4f688cbd1" is_pre_election: true
I20260809 15:07:13.656788 27736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:13.645827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:45523 (local address 127.24.183.193:51013)
0809 15:07:13.649714 (+ 3887us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:13.649729 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:13.649866 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:13.650184 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:13.650189 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:13.650213 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:07:13.650594 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:13.650603 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.652007 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.652013 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:13.654188 (+ 2175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:13.654204 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.656055 (+ 1851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.656060 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:13.656190 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:13.656214 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 15:07:13.656297 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 15:07:13.656417 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3734,"thread_start_us":3162,"threads_started":1}
I20260809 15:07:13.656778 27688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:13.645894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:51013 (local address 127.24.183.195:45523)
0809 15:07:13.646085 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:13.646093 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:13.649803 (+ 3710us) server_negotiation.cc:407] Connection header received
0809 15:07:13.649995 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:13.649999 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:13.650064 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:13.650141 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:07:13.650774 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.651851 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.654368 (+ 2517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.655603 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.655878 (+ 275us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:13.655931 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:13.656346 (+ 415us) server_negotiation.cc:299] Negotiation successful
0809 15:07:13.656484 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 15:07:13.656816 27397 raft_consensus.cc:2454] T 6016eada77a14f918a2c29c64593edf7 P ba2c864293ad499f8e13bee4f688cbd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be36dd1898cd4b008b8295d3923410bd in term 0.
I20260809 15:07:13.657392 27076 leader_election.cc:302] T 6016eada77a14f918a2c29c64593edf7 P be36dd1898cd4b008b8295d3923410bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2c864293ad499f8e13bee4f688cbd1, be36dd1898cd4b008b8295d3923410bd; no voters:
I20260809 15:07:13.657810 27701 raft_consensus.cc:2781] T 6016eada77a14f918a2c29c64593edf7 P be36dd1898cd4b008b8295d3923410bd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:07:13.657594 27598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6016eada77a14f918a2c29c64593edf7" candidate_uuid: "be36dd1898cd4b008b8295d3923410bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe6edff75377406780d04ee31436ddce" is_pre_election: true
I20260809 15:07:13.657928 27701 raft_consensus.cc:482] T 6016eada77a14f918a2c29c64593edf7 P be36dd1898cd4b008b8295d3923410bd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:07:13.658026 27701 raft_consensus.cc:3037] T 6016eada77a14f918a2c29c64593edf7 P be36dd1898cd4b008b8295d3923410bd [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:13.658114 27598 raft_consensus.cc:2454] T 6016eada77a14f918a2c29c64593edf7 P fe6edff75377406780d04ee31436ddce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be36dd1898cd4b008b8295d3923410bd in term 0.
I20260809 15:07:13.660789 27701 raft_consensus.cc:504] T 6016eada77a14f918a2c29c64593edf7 P be36dd1898cd4b008b8295d3923410bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } }
I20260809 15:07:13.661353 27701 leader_election.cc:288] T 6016eada77a14f918a2c29c64593edf7 P be36dd1898cd4b008b8295d3923410bd [CANDIDATE]: Term 1 election: Requested vote from peers ba2c864293ad499f8e13bee4f688cbd1 (127.24.183.194:41627), fe6edff75377406780d04ee31436ddce (127.24.183.195:45523)
I20260809 15:07:13.661834 27598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6016eada77a14f918a2c29c64593edf7" candidate_uuid: "be36dd1898cd4b008b8295d3923410bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe6edff75377406780d04ee31436ddce"
I20260809 15:07:13.661885 27397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6016eada77a14f918a2c29c64593edf7" candidate_uuid: "be36dd1898cd4b008b8295d3923410bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2c864293ad499f8e13bee4f688cbd1"
I20260809 15:07:13.662009 27598 raft_consensus.cc:3037] T 6016eada77a14f918a2c29c64593edf7 P fe6edff75377406780d04ee31436ddce [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:13.662125 27397 raft_consensus.cc:3037] T 6016eada77a14f918a2c29c64593edf7 P ba2c864293ad499f8e13bee4f688cbd1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:13.664362 27598 raft_consensus.cc:2454] T 6016eada77a14f918a2c29c64593edf7 P fe6edff75377406780d04ee31436ddce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be36dd1898cd4b008b8295d3923410bd in term 1.
I20260809 15:07:13.664736 27077 leader_election.cc:302] T 6016eada77a14f918a2c29c64593edf7 P be36dd1898cd4b008b8295d3923410bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be36dd1898cd4b008b8295d3923410bd, fe6edff75377406780d04ee31436ddce; no voters:
I20260809 15:07:13.665006 27701 raft_consensus.cc:2781] T 6016eada77a14f918a2c29c64593edf7 P be36dd1898cd4b008b8295d3923410bd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:07:13.665257 27397 raft_consensus.cc:2454] T 6016eada77a14f918a2c29c64593edf7 P ba2c864293ad499f8e13bee4f688cbd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be36dd1898cd4b008b8295d3923410bd in term 1.
I20260809 15:07:13.665582 27701 raft_consensus.cc:686] T 6016eada77a14f918a2c29c64593edf7 P be36dd1898cd4b008b8295d3923410bd [term 1 LEADER]: Becoming Leader. State: Replica: be36dd1898cd4b008b8295d3923410bd, State: Running, Role: LEADER
I20260809 15:07:13.666010 27701 consensus_queue.cc:237] T 6016eada77a14f918a2c29c64593edf7 P be36dd1898cd4b008b8295d3923410bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } }
I20260809 15:07:13.669313 26943 catalog_manager.cc:5495] T 6016eada77a14f918a2c29c64593edf7 P be36dd1898cd4b008b8295d3923410bd reported cstate change: term changed from 0 to 1, leader changed from <none> to be36dd1898cd4b008b8295d3923410bd (127.24.183.193). New cstate: current_term: 1 leader_uuid: "be36dd1898cd4b008b8295d3923410bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } health_report { overall_health: UNKNOWN } } }
Aug 09 15:07:13 dist-test-slave-nj21 krb5kdc[26887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288031, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:13.728839 27083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:13.714052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59840 (local address 127.24.183.254:42375)
0809 15:07:13.714276 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:13.714283 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:13.715036 (+ 753us) server_negotiation.cc:407] Connection header received
0809 15:07:13.715198 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:13.715205 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:13.715306 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:13.715432 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:13.716637 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.717803 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.719039 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.719822 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.719961 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:13.723494 (+ 3533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:13.723500 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:13.723503 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:13.723532 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:13.726014 (+ 2482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:13.726897 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:13.726901 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:13.726903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:13.727029 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:13.727455 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:13.727459 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:13.727461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:13.727768 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:13.727884 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:13.728408 (+ 524us) server_negotiation.cc:299] Negotiation successful
0809 15:07:13.728578 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 15:07:13.729293 27749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:13.714086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:42375 (local address 127.0.0.1:59840)
0809 15:07:13.714774 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:13.714793 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:13.715081 (+ 288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:13.715457 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:13.715460 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:13.716196 (+ 736us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:13.716481 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:13.716490 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.717926 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.717929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:13.718906 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:13.718917 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.719996 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.720001 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:13.720126 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:13.720739 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:13.720761 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:13.723317 (+ 2556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:13.726143 (+ 2826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:13.726157 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:13.726165 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:13.726771 (+ 606us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:13.727134 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:13.727139 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:13.727144 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:13.727350 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:13.727891 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:13.727900 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:13.728125 (+ 225us) client_negotiation.cc:769] Sending connection context
0809 15:07:13.728342 (+ 217us) client_negotiation.cc:240] Negotiation successful
0809 15:07:13.728750 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"mutex_wait_us":100,"thread_start_us":386,"threads_started":1}
I20260809 15:07:13.734869 26943 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:59840:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 15:07:13.736481 26943 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:07:13.743664 27365 tablet_service.cc:1467] Processing CreateTablet for tablet 228b2ee913614f298464d1111c6a8571 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=436743ca8cef4b3c82424808982d69e2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 15:07:13.743641 27150 tablet_service.cc:1467] Processing CreateTablet for tablet 228b2ee913614f298464d1111c6a8571 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=436743ca8cef4b3c82424808982d69e2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 15:07:13.744263 27365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 228b2ee913614f298464d1111c6a8571. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:13.744287 27150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 228b2ee913614f298464d1111c6a8571. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:13.747826 27561 tablet_service.cc:1467] Processing CreateTablet for tablet 228b2ee913614f298464d1111c6a8571 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=436743ca8cef4b3c82424808982d69e2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 15:07:13.748452 27561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 228b2ee913614f298464d1111c6a8571. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:13.751609 27695 tablet_bootstrap.cc:492] T 228b2ee913614f298464d1111c6a8571 P ba2c864293ad499f8e13bee4f688cbd1: Bootstrap starting.
I20260809 15:07:13.753321 27695 tablet_bootstrap.cc:654] T 228b2ee913614f298464d1111c6a8571 P ba2c864293ad499f8e13bee4f688cbd1: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:13.754994 27696 tablet_bootstrap.cc:492] T 228b2ee913614f298464d1111c6a8571 P be36dd1898cd4b008b8295d3923410bd: Bootstrap starting.
I20260809 15:07:13.756357 27691 tablet_bootstrap.cc:492] T 228b2ee913614f298464d1111c6a8571 P fe6edff75377406780d04ee31436ddce: Bootstrap starting.
I20260809 15:07:13.756784 27696 tablet_bootstrap.cc:654] T 228b2ee913614f298464d1111c6a8571 P be36dd1898cd4b008b8295d3923410bd: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:13.758126 27691 tablet_bootstrap.cc:654] T 228b2ee913614f298464d1111c6a8571 P fe6edff75377406780d04ee31436ddce: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:13.763875 27691 tablet_bootstrap.cc:492] T 228b2ee913614f298464d1111c6a8571 P fe6edff75377406780d04ee31436ddce: No bootstrap required, opened a new log
I20260809 15:07:13.764050 27691 ts_tablet_manager.cc:1397] T 228b2ee913614f298464d1111c6a8571 P fe6edff75377406780d04ee31436ddce: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.002s
I20260809 15:07:13.764725 27696 tablet_bootstrap.cc:492] T 228b2ee913614f298464d1111c6a8571 P be36dd1898cd4b008b8295d3923410bd: No bootstrap required, opened a new log
I20260809 15:07:13.764745 27695 tablet_bootstrap.cc:492] T 228b2ee913614f298464d1111c6a8571 P ba2c864293ad499f8e13bee4f688cbd1: No bootstrap required, opened a new log
I20260809 15:07:13.764907 27696 ts_tablet_manager.cc:1397] T 228b2ee913614f298464d1111c6a8571 P be36dd1898cd4b008b8295d3923410bd: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.009s
I20260809 15:07:13.764914 27695 ts_tablet_manager.cc:1397] T 228b2ee913614f298464d1111c6a8571 P ba2c864293ad499f8e13bee4f688cbd1: Time spent bootstrapping tablet: real 0.013s user 0.011s sys 0.000s
I20260809 15:07:13.764900 27691 raft_consensus.cc:348] T 228b2ee913614f298464d1111c6a8571 P fe6edff75377406780d04ee31436ddce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } }
I20260809 15:07:13.765143 27691 raft_consensus.cc:374] T 228b2ee913614f298464d1111c6a8571 P fe6edff75377406780d04ee31436ddce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:13.765229 27691 raft_consensus.cc:729] T 228b2ee913614f298464d1111c6a8571 P fe6edff75377406780d04ee31436ddce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe6edff75377406780d04ee31436ddce, State: Initialized, Role: FOLLOWER
I20260809 15:07:13.765491 27691 consensus_queue.cc:260] T 228b2ee913614f298464d1111c6a8571 P fe6edff75377406780d04ee31436ddce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } }
I20260809 15:07:13.765668 27695 raft_consensus.cc:348] T 228b2ee913614f298464d1111c6a8571 P ba2c864293ad499f8e13bee4f688cbd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } }
I20260809 15:07:13.765764 27696 raft_consensus.cc:348] T 228b2ee913614f298464d1111c6a8571 P be36dd1898cd4b008b8295d3923410bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } }
I20260809 15:07:13.765928 27695 raft_consensus.cc:374] T 228b2ee913614f298464d1111c6a8571 P ba2c864293ad499f8e13bee4f688cbd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:13.765998 27696 raft_consensus.cc:374] T 228b2ee913614f298464d1111c6a8571 P be36dd1898cd4b008b8295d3923410bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:13.766008 27695 raft_consensus.cc:729] T 228b2ee913614f298464d1111c6a8571 P ba2c864293ad499f8e13bee4f688cbd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2c864293ad499f8e13bee4f688cbd1, State: Initialized, Role: FOLLOWER
I20260809 15:07:13.766042 27691 ts_tablet_manager.cc:1428] T 228b2ee913614f298464d1111c6a8571 P fe6edff75377406780d04ee31436ddce: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 15:07:13.766072 27696 raft_consensus.cc:729] T 228b2ee913614f298464d1111c6a8571 P be36dd1898cd4b008b8295d3923410bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be36dd1898cd4b008b8295d3923410bd, State: Initialized, Role: FOLLOWER
I20260809 15:07:13.766240 27695 consensus_queue.cc:260] T 228b2ee913614f298464d1111c6a8571 P ba2c864293ad499f8e13bee4f688cbd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } }
I20260809 15:07:13.766350 27696 consensus_queue.cc:260] T 228b2ee913614f298464d1111c6a8571 P be36dd1898cd4b008b8295d3923410bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } }
I20260809 15:07:13.766914 27701 tablet_replica.cc:439] T 228b2ee913614f298464d1111c6a8571 P be36dd1898cd4b008b8295d3923410bd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } } }
I20260809 15:07:13.766875 27702 tablet_replica.cc:439] T 228b2ee913614f298464d1111c6a8571 P ba2c864293ad499f8e13bee4f688cbd1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } } }
I20260809 15:07:13.767213 27702 tablet_replica.cc:442] T 228b2ee913614f298464d1111c6a8571 P ba2c864293ad499f8e13bee4f688cbd1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:07:13.767258 27701 tablet_replica.cc:442] T 228b2ee913614f298464d1111c6a8571 P be36dd1898cd4b008b8295d3923410bd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:07:13.766940 27696 ts_tablet_manager.cc:1428] T 228b2ee913614f298464d1111c6a8571 P be36dd1898cd4b008b8295d3923410bd: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 15:07:13.768157 27695 ts_tablet_manager.cc:1428] T 228b2ee913614f298464d1111c6a8571 P ba2c864293ad499f8e13bee4f688cbd1: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 15:07:13.768716 27715 tablet_replica.cc:439] T 228b2ee913614f298464d1111c6a8571 P fe6edff75377406780d04ee31436ddce: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } } }
I20260809 15:07:13.769095 27715 tablet_replica.cc:442] T 228b2ee913614f298464d1111c6a8571 P fe6edff75377406780d04ee31436ddce: This TxnStatusTablet replica's current role is: FOLLOWER
W20260809 15:07:13.807520 27668 tablet.cc:2369] T 228b2ee913614f298464d1111c6a8571 P fe6edff75377406780d04ee31436ddce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:07:13.912911 27715 raft_consensus.cc:482] T 228b2ee913614f298464d1111c6a8571 P fe6edff75377406780d04ee31436ddce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:07:13.913172 27715 raft_consensus.cc:504] T 228b2ee913614f298464d1111c6a8571 P fe6edff75377406780d04ee31436ddce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } }
I20260809 15:07:13.913898 27715 leader_election.cc:288] T 228b2ee913614f298464d1111c6a8571 P fe6edff75377406780d04ee31436ddce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be36dd1898cd4b008b8295d3923410bd (127.24.183.193:40361), ba2c864293ad499f8e13bee4f688cbd1 (127.24.183.194:41627)
I20260809 15:07:13.914518 27397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "228b2ee913614f298464d1111c6a8571" candidate_uuid: "fe6edff75377406780d04ee31436ddce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2c864293ad499f8e13bee4f688cbd1" is_pre_election: true
I20260809 15:07:13.914556 27183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "228b2ee913614f298464d1111c6a8571" candidate_uuid: "fe6edff75377406780d04ee31436ddce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be36dd1898cd4b008b8295d3923410bd" is_pre_election: true
I20260809 15:07:13.914804 27397 raft_consensus.cc:2454] T 228b2ee913614f298464d1111c6a8571 P ba2c864293ad499f8e13bee4f688cbd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe6edff75377406780d04ee31436ddce in term 0.
I20260809 15:07:13.914899 27183 raft_consensus.cc:2454] T 228b2ee913614f298464d1111c6a8571 P be36dd1898cd4b008b8295d3923410bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe6edff75377406780d04ee31436ddce in term 0.
I20260809 15:07:13.915283 27491 leader_election.cc:302] T 228b2ee913614f298464d1111c6a8571 P fe6edff75377406780d04ee31436ddce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2c864293ad499f8e13bee4f688cbd1, fe6edff75377406780d04ee31436ddce; no voters:
I20260809 15:07:13.915604 27715 raft_consensus.cc:2781] T 228b2ee913614f298464d1111c6a8571 P fe6edff75377406780d04ee31436ddce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:07:13.915802 27715 raft_consensus.cc:482] T 228b2ee913614f298464d1111c6a8571 P fe6edff75377406780d04ee31436ddce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:07:13.915921 27715 raft_consensus.cc:3037] T 228b2ee913614f298464d1111c6a8571 P fe6edff75377406780d04ee31436ddce [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:13.919241 27715 raft_consensus.cc:504] T 228b2ee913614f298464d1111c6a8571 P fe6edff75377406780d04ee31436ddce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } }
I20260809 15:07:13.919935 27715 leader_election.cc:288] T 228b2ee913614f298464d1111c6a8571 P fe6edff75377406780d04ee31436ddce [CANDIDATE]: Term 1 election: Requested vote from peers be36dd1898cd4b008b8295d3923410bd (127.24.183.193:40361), ba2c864293ad499f8e13bee4f688cbd1 (127.24.183.194:41627)
I20260809 15:07:13.920377 27397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "228b2ee913614f298464d1111c6a8571" candidate_uuid: "fe6edff75377406780d04ee31436ddce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba2c864293ad499f8e13bee4f688cbd1"
I20260809 15:07:13.920413 27183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "228b2ee913614f298464d1111c6a8571" candidate_uuid: "fe6edff75377406780d04ee31436ddce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be36dd1898cd4b008b8295d3923410bd"
I20260809 15:07:13.920586 27397 raft_consensus.cc:3037] T 228b2ee913614f298464d1111c6a8571 P ba2c864293ad499f8e13bee4f688cbd1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:13.920601 27183 raft_consensus.cc:3037] T 228b2ee913614f298464d1111c6a8571 P be36dd1898cd4b008b8295d3923410bd [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:13.922847 27397 raft_consensus.cc:2454] T 228b2ee913614f298464d1111c6a8571 P ba2c864293ad499f8e13bee4f688cbd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe6edff75377406780d04ee31436ddce in term 1.
I20260809 15:07:13.923221 27491 leader_election.cc:302] T 228b2ee913614f298464d1111c6a8571 P fe6edff75377406780d04ee31436ddce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2c864293ad499f8e13bee4f688cbd1, fe6edff75377406780d04ee31436ddce; no voters:
I20260809 15:07:13.923491 27715 raft_consensus.cc:2781] T 228b2ee913614f298464d1111c6a8571 P fe6edff75377406780d04ee31436ddce [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:07:13.923650 27715 raft_consensus.cc:686] T 228b2ee913614f298464d1111c6a8571 P fe6edff75377406780d04ee31436ddce [term 1 LEADER]: Becoming Leader. State: Replica: fe6edff75377406780d04ee31436ddce, State: Running, Role: LEADER
I20260809 15:07:13.923640 27183 raft_consensus.cc:2454] T 228b2ee913614f298464d1111c6a8571 P be36dd1898cd4b008b8295d3923410bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe6edff75377406780d04ee31436ddce in term 1.
I20260809 15:07:13.923998 27715 consensus_queue.cc:237] T 228b2ee913614f298464d1111c6a8571 P fe6edff75377406780d04ee31436ddce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } }
I20260809 15:07:13.925966 27705 tablet_replica.cc:439] T 228b2ee913614f298464d1111c6a8571 P fe6edff75377406780d04ee31436ddce: TxnStatusTablet state changed. Reason: New leader fe6edff75377406780d04ee31436ddce. Latest consensus state: current_term: 1 leader_uuid: "fe6edff75377406780d04ee31436ddce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } } }
I20260809 15:07:13.926343 27705 tablet_replica.cc:442] T 228b2ee913614f298464d1111c6a8571 P fe6edff75377406780d04ee31436ddce: This TxnStatusTablet replica's current role is: LEADER
I20260809 15:07:13.927536 26942 catalog_manager.cc:5495] T 228b2ee913614f298464d1111c6a8571 P fe6edff75377406780d04ee31436ddce reported cstate change: term changed from 0 to 1, leader changed from <none> to fe6edff75377406780d04ee31436ddce (127.24.183.195). New cstate: current_term: 1 leader_uuid: "fe6edff75377406780d04ee31436ddce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } health_report { overall_health: HEALTHY } } }
I20260809 15:07:13.927989 27755 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 15:07:13.928303 27755 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 15:07:13.949033 27749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:13.944138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:45523 (local address 127.0.0.1:52438)
0809 15:07:13.944346 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:13.944363 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:13.944491 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:13.944860 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:13.944865 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:13.944877 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 15:07:13.945049 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:13.945058 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.946066 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.946071 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:13.947071 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:13.947084 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.947892 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.947898 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:13.948049 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:13.948103 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 15:07:13.948486 (+ 383us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 15:07:13.948507 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 15:07:13.948577 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 15:07:13.948726 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 15:07:13.949023 27688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:13.944352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52438 (local address 127.24.183.195:45523)
0809 15:07:13.944559 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:13.944565 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:13.944584 (+ 19us) server_negotiation.cc:407] Connection header received
0809 15:07:13.944668 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:13.944673 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:13.944740 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:13.944817 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:07:13.945180 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.945940 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.947209 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.947745 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.947870 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:13.948278 (+ 408us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:07:13.948394 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:07:13.948467 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:13.948616 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 15:07:13.948692 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 15:07:13.953224 27183 raft_consensus.cc:1261] T 228b2ee913614f298464d1111c6a8571 P be36dd1898cd4b008b8295d3923410bd [term 1 FOLLOWER]: Refusing update from remote peer fe6edff75377406780d04ee31436ddce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:07:13.953348 27397 raft_consensus.cc:1261] T 228b2ee913614f298464d1111c6a8571 P ba2c864293ad499f8e13bee4f688cbd1 [term 1 FOLLOWER]: Refusing update from remote peer fe6edff75377406780d04ee31436ddce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:07:13.953893 27705 consensus_queue.cc:1035] T 228b2ee913614f298464d1111c6a8571 P fe6edff75377406780d04ee31436ddce [LEADER]: Connected to new peer: Peer: permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:07:13.954213 27715 consensus_queue.cc:1035] T 228b2ee913614f298464d1111c6a8571 P fe6edff75377406780d04ee31436ddce [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:07:13.958531 27702 tablet_replica.cc:439] T 228b2ee913614f298464d1111c6a8571 P ba2c864293ad499f8e13bee4f688cbd1: TxnStatusTablet state changed. Reason: New leader fe6edff75377406780d04ee31436ddce. Latest consensus state: current_term: 1 leader_uuid: "fe6edff75377406780d04ee31436ddce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } } }
I20260809 15:07:13.958827 27702 tablet_replica.cc:442] T 228b2ee913614f298464d1111c6a8571 P ba2c864293ad499f8e13bee4f688cbd1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:07:13.959600 27705 tablet_replica.cc:439] T 228b2ee913614f298464d1111c6a8571 P fe6edff75377406780d04ee31436ddce: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fe6edff75377406780d04ee31436ddce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } } }
I20260809 15:07:13.959849 27705 tablet_replica.cc:442] T 228b2ee913614f298464d1111c6a8571 P fe6edff75377406780d04ee31436ddce: This TxnStatusTablet replica's current role is: LEADER
I20260809 15:07:13.961632 27702 tablet_replica.cc:439] T 228b2ee913614f298464d1111c6a8571 P ba2c864293ad499f8e13bee4f688cbd1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fe6edff75377406780d04ee31436ddce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } } }
I20260809 15:07:13.961902 27702 tablet_replica.cc:442] T 228b2ee913614f298464d1111c6a8571 P ba2c864293ad499f8e13bee4f688cbd1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:07:13.961937 27710 mvcc.cc:205] Tried to move back new op lower bound from 7316635787066265600 to 7316635786955071488. Current Snapshot: MvccSnapshot[applied={T|T < 7316635787066265600 or (T in {7316635787066265600})}]
I20260809 15:07:13.957716 27701 tablet_replica.cc:439] T 228b2ee913614f298464d1111c6a8571 P be36dd1898cd4b008b8295d3923410bd: TxnStatusTablet state changed. Reason: New leader fe6edff75377406780d04ee31436ddce. Latest consensus state: current_term: 1 leader_uuid: "fe6edff75377406780d04ee31436ddce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } } }
I20260809 15:07:13.963516 27701 tablet_replica.cc:442] T 228b2ee913614f298464d1111c6a8571 P be36dd1898cd4b008b8295d3923410bd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:07:13.966322 27715 tablet_replica.cc:439] T 228b2ee913614f298464d1111c6a8571 P fe6edff75377406780d04ee31436ddce: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fe6edff75377406780d04ee31436ddce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } } }
I20260809 15:07:13.966603 27715 tablet_replica.cc:442] T 228b2ee913614f298464d1111c6a8571 P fe6edff75377406780d04ee31436ddce: This TxnStatusTablet replica's current role is: LEADER
I20260809 15:07:13.968660 27711 mvcc.cc:205] Tried to move back new op lower bound from 7316635787066265600 to 7316635786955071488. Current Snapshot: MvccSnapshot[applied={T|T < 7316635787066265600}]
I20260809 15:07:13.968955 27701 tablet_replica.cc:439] T 228b2ee913614f298464d1111c6a8571 P be36dd1898cd4b008b8295d3923410bd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fe6edff75377406780d04ee31436ddce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be36dd1898cd4b008b8295d3923410bd" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 40361 } } peers { permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 } } peers { permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 } } }
I20260809 15:07:13.969237 27701 tablet_replica.cc:442] T 228b2ee913614f298464d1111c6a8571 P be36dd1898cd4b008b8295d3923410bd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:07:13.977319 27760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:13.971613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58070 (local address 127.24.183.193:40361)
0809 15:07:13.972085 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:13.972090 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:13.972104 (+ 14us) server_negotiation.cc:407] Connection header received
0809 15:07:13.972181 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:13.972185 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:13.972269 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:13.972408 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:07:13.972799 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.973559 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.974854 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.975264 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.975369 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:13.975838 (+ 469us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:07:13.976589 (+ 751us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:07:13.976759 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:13.976870 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 15:07:13.976958 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":268,"threads_started":1}
I20260809 15:07:13.994594 27688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:13.988229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:48981 (local address 127.24.183.195:45523)
0809 15:07:13.988383 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:13.988387 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:13.988969 (+ 582us) server_negotiation.cc:407] Connection header received
0809 15:07:13.989150 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:13.989154 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:13.989272 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:13.989374 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:07:13.989973 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.990722 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.992790 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.993334 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.993463 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:13.993973 (+ 510us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:07:13.994082 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:07:13.994152 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:13.994308 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 15:07:13.994381 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 15:07:13.994843 27764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:13.988237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:45523 (local address 127.24.183.193:48981)
0809 15:07:13.988860 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:13.988879 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:13.989014 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:13.989422 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:13.989428 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:13.989455 (+ 27us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 15:07:13.989814 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:13.989828 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.990848 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.990853 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:13.992631 (+ 1778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:13.992647 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:13.993541 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:13.993547 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:13.993726 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:13.993799 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 15:07:13.994182 (+ 383us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 15:07:13.994195 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 15:07:13.994299 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 15:07:13.994451 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":329,"threads_started":1}
I20260809 15:07:14.005503 27397 raft_consensus.cc:1261] T 6016eada77a14f918a2c29c64593edf7 P ba2c864293ad499f8e13bee4f688cbd1 [term 1 FOLLOWER]: Refusing update from remote peer be36dd1898cd4b008b8295d3923410bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:07:14.005616 27598 raft_consensus.cc:1261] T 6016eada77a14f918a2c29c64593edf7 P fe6edff75377406780d04ee31436ddce [term 1 FOLLOWER]: Refusing update from remote peer be36dd1898cd4b008b8295d3923410bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:07:14.006191 27739 consensus_queue.cc:1035] T 6016eada77a14f918a2c29c64593edf7 P be36dd1898cd4b008b8295d3923410bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2c864293ad499f8e13bee4f688cbd1" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 41627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:07:14.006549 27701 consensus_queue.cc:1035] T 6016eada77a14f918a2c29c64593edf7 P be36dd1898cd4b008b8295d3923410bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe6edff75377406780d04ee31436ddce" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:07:14.043768 27769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:14.037332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:40361 (local address 127.24.183.195:53497)
0809 15:07:14.037996 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:14.038016 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:14.038167 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:14.038471 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:14.038476 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:14.038490 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 15:07:14.038897 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:14.038908 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:14.040063 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:14.040069 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:14.041609 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:14.041622 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:14.042405 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:14.042412 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:14.042584 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:14.042640 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 15:07:14.043093 (+ 453us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 15:07:14.043110 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 15:07:14.043180 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 15:07:14.043320 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":572,"threads_started":1}
I20260809 15:07:14.043840 27760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:14.037721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:53497 (local address 127.24.183.193:40361)
0809 15:07:14.037893 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:14.037899 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:14.038108 (+ 209us) server_negotiation.cc:407] Connection header received
0809 15:07:14.038288 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:14.038291 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:14.038355 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:14.038430 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:07:14.039066 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:14.039920 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:14.041747 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:14.042227 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:14.042338 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:14.042807 (+ 469us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:07:14.042962 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:07:14.043047 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:14.043463 (+ 416us) server_negotiation.cc:299] Negotiation successful
0809 15:07:14.043548 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 15:07:14.083760 26942 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:07:14.084040 26942 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:07:14.087438 26942 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a7c251fa7fa546fc86a1d5e92878800e: Sending DeleteTablet for 3 replicas of tablet 6016eada77a14f918a2c29c64593edf7
I20260809 15:07:14.088274 27150 tablet_service.cc:1514] Processing DeleteTablet for tablet 6016eada77a14f918a2c29c64593edf7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:07:14 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:58058
I20260809 15:07:14.088593 27728 tablet_replica.cc:330] T 6016eada77a14f918a2c29c64593edf7 P be36dd1898cd4b008b8295d3923410bd: stopping tablet replica
I20260809 15:07:14.088864 27728 raft_consensus.cc:2229] T 6016eada77a14f918a2c29c64593edf7 P be36dd1898cd4b008b8295d3923410bd [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:07:14.088867 27365 tablet_service.cc:1514] Processing DeleteTablet for tablet 6016eada77a14f918a2c29c64593edf7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:07:14 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:52992
I20260809 15:07:14.089139 27729 tablet_replica.cc:330] T 6016eada77a14f918a2c29c64593edf7 P ba2c864293ad499f8e13bee4f688cbd1: stopping tablet replica
I20260809 15:07:14.089361 27729 raft_consensus.cc:2229] T 6016eada77a14f918a2c29c64593edf7 P ba2c864293ad499f8e13bee4f688cbd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:07:14.089457 27728 raft_consensus.cc:2258] T 6016eada77a14f918a2c29c64593edf7 P be36dd1898cd4b008b8295d3923410bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:07:14.089592 27729 raft_consensus.cc:2258] T 6016eada77a14f918a2c29c64593edf7 P ba2c864293ad499f8e13bee4f688cbd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:07:14.090648 27561 tablet_service.cc:1514] Processing DeleteTablet for tablet 6016eada77a14f918a2c29c64593edf7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:07:14 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:52412
I20260809 15:07:14.090862 27728 ts_tablet_manager.cc:1905] T 6016eada77a14f918a2c29c64593edf7 P be36dd1898cd4b008b8295d3923410bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:07:14.090981 27721 tablet_replica.cc:330] T 6016eada77a14f918a2c29c64593edf7 P fe6edff75377406780d04ee31436ddce: stopping tablet replica
I20260809 15:07:14.091194 27721 raft_consensus.cc:2229] T 6016eada77a14f918a2c29c64593edf7 P fe6edff75377406780d04ee31436ddce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:07:14.091409 27721 raft_consensus.cc:2258] T 6016eada77a14f918a2c29c64593edf7 P fe6edff75377406780d04ee31436ddce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:07:14.091734 27729 ts_tablet_manager.cc:1905] T 6016eada77a14f918a2c29c64593edf7 P ba2c864293ad499f8e13bee4f688cbd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:07:14.092788 27721 ts_tablet_manager.cc:1905] T 6016eada77a14f918a2c29c64593edf7 P fe6edff75377406780d04ee31436ddce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:07:14.098719 27729 ts_tablet_manager.cc:1918] T 6016eada77a14f918a2c29c64593edf7 P ba2c864293ad499f8e13bee4f688cbd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:07:14.098868 27729 log.cc:1196] T 6016eada77a14f918a2c29c64593edf7 P ba2c864293ad499f8e13bee4f688cbd1: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-1/wal/wals/6016eada77a14f918a2c29c64593edf7
I20260809 15:07:14.099325 27729 ts_tablet_manager.cc:1939] T 6016eada77a14f918a2c29c64593edf7 P ba2c864293ad499f8e13bee4f688cbd1: Deleting consensus metadata
I20260809 15:07:14.100165 27721 ts_tablet_manager.cc:1918] T 6016eada77a14f918a2c29c64593edf7 P fe6edff75377406780d04ee31436ddce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:07:14.100381 27721 log.cc:1196] T 6016eada77a14f918a2c29c64593edf7 P fe6edff75377406780d04ee31436ddce: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-2/wal/wals/6016eada77a14f918a2c29c64593edf7
I20260809 15:07:14.100458 26925 catalog_manager.cc:4841] TS ba2c864293ad499f8e13bee4f688cbd1 (127.24.183.194:41627): tablet 6016eada77a14f918a2c29c64593edf7 (table test-table [id=94cb9ec198534d64a4d2ae7e2260e433]) successfully deleted
I20260809 15:07:14.100852 27721 ts_tablet_manager.cc:1939] T 6016eada77a14f918a2c29c64593edf7 P fe6edff75377406780d04ee31436ddce: Deleting consensus metadata
I20260809 15:07:14.101938 26926 catalog_manager.cc:4841] TS fe6edff75377406780d04ee31436ddce (127.24.183.195:45523): tablet 6016eada77a14f918a2c29c64593edf7 (table test-table [id=94cb9ec198534d64a4d2ae7e2260e433]) successfully deleted
I20260809 15:07:14.103820 27728 ts_tablet_manager.cc:1918] T 6016eada77a14f918a2c29c64593edf7 P be36dd1898cd4b008b8295d3923410bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:07:14.104737 27728 log.cc:1196] T 6016eada77a14f918a2c29c64593edf7 P be36dd1898cd4b008b8295d3923410bd: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288021313853-25311-0/minicluster-data/ts-0/wal/wals/6016eada77a14f918a2c29c64593edf7
I20260809 15:07:14.105314 27728 ts_tablet_manager.cc:1939] T 6016eada77a14f918a2c29c64593edf7 P be36dd1898cd4b008b8295d3923410bd: Deleting consensus metadata
I20260809 15:07:14.110052 26923 catalog_manager.cc:4841] TS be36dd1898cd4b008b8295d3923410bd (127.24.183.193:40361): tablet 6016eada77a14f918a2c29c64593edf7 (table test-table [id=94cb9ec198534d64a4d2ae7e2260e433]) successfully deleted
Aug 09 15:07:14 dist-test-slave-nj21 krb5kdc[26887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288033, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.183.193@KRBTEST.COM
I20260809 15:07:14.140825 27760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:14.104115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58072 (local address 127.24.183.193:40361)
0809 15:07:14.104348 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:14.104355 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:14.111152 (+ 6797us) server_negotiation.cc:407] Connection header received
0809 15:07:14.111469 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:14.111475 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:14.111565 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:14.111653 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:14.114481 (+ 2828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:14.116251 (+ 1770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:14.119902 (+ 3651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:14.120979 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:14.121080 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:14.130389 (+ 9309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:14.130413 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:14.130438 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:14.130489 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:14.135892 (+ 5403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:14.137107 (+ 1215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:14.137117 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:14.137121 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:14.137436 (+ 315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:14.138058 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:14.138064 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:14.138067 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:14.138620 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:14.138945 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:14.139874 (+ 929us) server_negotiation.cc:299] Negotiation successful
0809 15:07:14.140140 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260809 15:07:14.144029 27246 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58072
I20260809 15:07:14.179052 27083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:14.160196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59850 (local address 127.24.183.254:42375)
0809 15:07:14.160478 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:14.160486 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:14.161492 (+ 1006us) server_negotiation.cc:407] Connection header received
0809 15:07:14.161664 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:14.161671 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:14.161780 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:14.161909 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:14.164624 (+ 2715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:14.165749 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:14.167851 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:14.168561 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:14.168662 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:14.171972 (+ 3310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:14.171980 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:14.171985 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:14.172026 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:14.175588 (+ 3562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:14.176466 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:14.176472 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:14.176476 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:14.176626 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:14.177102 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:14.177107 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:14.177110 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:14.177478 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:14.177757 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:14.178419 (+ 662us) server_negotiation.cc:299] Negotiation successful
0809 15:07:14.178660 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
W20260809 15:07:14.180572 26942 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59850
I20260809 15:07:14.183784 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 27049
I20260809 15:07:14.202118 27760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:14.185044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58078 (local address 127.24.183.193:40361)
0809 15:07:14.185244 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:14.185250 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:14.186533 (+ 1283us) server_negotiation.cc:407] Connection header received
0809 15:07:14.186670 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:14.186676 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:14.186783 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:14.186886 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:14.188370 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:14.189482 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:14.191221 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:14.192046 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:14.192197 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:14.195265 (+ 3068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:14.195272 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:14.195277 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:14.195322 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:14.198957 (+ 3635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:14.199759 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:14.199765 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:14.199769 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:14.199929 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:14.200308 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:14.200313 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:14.200317 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:14.200672 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:14.200946 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:14.201230 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 15:07:14.201506 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260809 15:07:14.202843 27246 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58078
W20260809 15:07:14.203466 25311 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 27049: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:07:14.203647 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 27049
I20260809 15:07:14.231196 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 27259
Aug 09 15:07:14 dist-test-slave-nj21 krb5kdc[26887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288033, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.183.194@KRBTEST.COM
I20260809 15:07:14.247815 27787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:14.232010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53002 (local address 127.24.183.194:41627)
0809 15:07:14.232511 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:14.232515 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:14.232531 (+ 16us) server_negotiation.cc:407] Connection header received
0809 15:07:14.232585 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:14.232589 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:14.232658 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:14.232770 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:14.233958 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:14.234666 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:14.236093 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:14.236534 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:14.236631 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:14.241321 (+ 4690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:14.241340 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:14.241362 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:14.241407 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:14.245062 (+ 3655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:14.245681 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:14.245691 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:14.245695 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:14.245871 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:14.246178 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:14.246183 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:14.246187 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:14.246636 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:14.246944 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:14.247138 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 15:07:14.247372 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":273,"threads_started":1}
W20260809 15:07:14.248620 27457 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53002
W20260809 15:07:14.249272 25311 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 27259: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:07:14.249477 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 27259
I20260809 15:07:14.273681 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 27468
Aug 09 15:07:14 dist-test-slave-nj21 krb5kdc[26887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288033, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.183.195@KRBTEST.COM
I20260809 15:07:14.290848 27688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:14.274443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52450 (local address 127.24.183.195:45523)
0809 15:07:14.274627 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:14.274631 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:14.274652 (+ 21us) server_negotiation.cc:407] Connection header received
0809 15:07:14.274862 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:14.274866 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:14.274943 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:14.275023 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:14.276696 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:14.277476 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:14.279319 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:14.279944 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:14.280042 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:14.285127 (+ 5085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:14.285142 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:14.285160 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:14.285194 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:14.288233 (+ 3039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:14.288971 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:14.288978 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:14.288980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:14.289113 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:14.289479 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:14.289483 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:14.289485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:14.289845 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:14.290109 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:14.290371 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 15:07:14.290550 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260809 15:07:14.291538 27660 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52450
W20260809 15:07:14.292174 25311 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 27468: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:07:14.292307 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 27468
I20260809 15:07:14.320950 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 26902
I20260809 15:07:14.322041 27003 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:07:14.440146 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 26902
2026-08-09T15:07:14Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4610 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:07:14 dist-test-slave-nj21 krb5kdc[27791](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:07:14 dist-test-slave-nj21 krb5kdc[27791](info): set up 2 sockets
Aug 09 15:07:14 dist-test-slave-nj21 krb5kdc[27791](info): commencing operation
krb5kdc: starting...
W20260809 15:07:15.626963 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.001s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:07:15 dist-test-slave-nj21 krb5kdc[27791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:07:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:07:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
I20260809 15:07:15.812211 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:40281
--webserver_interface=127.24.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:38895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:40281
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:16.132890 27807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:16.133179 27807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:16.133322 27807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:16.140607 27807 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:07:16.140720 27807 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 15:07:16.140769 27807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:16.140863 27807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:16.140924 27807 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:07:16.141011 27807 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:07:16.150892 27807 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:38895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:40281
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:40281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:16.152235 27807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:16.153574 27807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:16.165197 27817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:16.165827 27818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:16.165848 27823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:16.166326 27807 server_base.cc:1034] running on GCE node
I20260809 15:07:16.167107 27807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:16.168493 27807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:16.169727 27807 hybrid_clock.cc:648] HybridClock initialized: now 1786288036169643 us; error 33 us; skew 500 ppm
I20260809 15:07:16.170040 27807 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:16 dist-test-slave-nj21 krb5kdc[27791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288036, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:16.173098 27807 init.cc:377] Logged in from keytab as kudu/127.24.183.254@KRBTEST.COM (short username kudu)
I20260809 15:07:16.174431 27807 webserver.cc:460] Webserver started at http://127.24.183.254:43983/ using document root <none> and password file <none>
I20260809 15:07:16.175099 27807 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:16.175192 27807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:16.175493 27807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:16.178454 27807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "9b8c4d0b7e114003ab0f7e0a059f798f"
format_stamp: "Formatted at 2026-08-09 15:07:16 on dist-test-slave-nj21"
server_key: "40d03ae5e7be07775a9dade1ba6a2546"
server_key_iv: "083bdb85cb146b0ff358b2de9ea14a13"
server_key_version: "encryptionkey@0"
I20260809 15:07:16.179225 27807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "9b8c4d0b7e114003ab0f7e0a059f798f"
format_stamp: "Formatted at 2026-08-09 15:07:16 on dist-test-slave-nj21"
server_key: "40d03ae5e7be07775a9dade1ba6a2546"
server_key_iv: "083bdb85cb146b0ff358b2de9ea14a13"
server_key_version: "encryptionkey@0"
I20260809 15:07:16.184326 27807 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260809 15:07:16.187747 27835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:16.189033 27807 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:07:16.189193 27807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "9b8c4d0b7e114003ab0f7e0a059f798f"
format_stamp: "Formatted at 2026-08-09 15:07:16 on dist-test-slave-nj21"
server_key: "40d03ae5e7be07775a9dade1ba6a2546"
server_key_iv: "083bdb85cb146b0ff358b2de9ea14a13"
server_key_version: "encryptionkey@0"
I20260809 15:07:16.189390 27807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:16.207571 27807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:16.211665 27807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:16.211939 27807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:16.233106 27807 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:40281
I20260809 15:07:16.233119 27931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:40281 every 8 connection(s)
I20260809 15:07:16.234395 27807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:07:16.238150 27932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:16.238757 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 27807
I20260809 15:07:16.238986 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:07:16.239840 25311 external_mini_cluster.cc:1428] Setting key 6afa10cfcd942d5d70b787cb90400f6c
I20260809 15:07:16.252532 27932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b8c4d0b7e114003ab0f7e0a059f798f: Bootstrap starting.
Aug 09 15:07:16 dist-test-slave-nj21 krb5kdc[27791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:16.256299 27932 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b8c4d0b7e114003ab0f7e0a059f798f: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:16.257369 27932 log.cc:824] T 00000000000000000000000000000000 P 9b8c4d0b7e114003ab0f7e0a059f798f: Log is configured to *not* fsync() on all Append() calls
I20260809 15:07:16.260608 27932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b8c4d0b7e114003ab0f7e0a059f798f: No bootstrap required, opened a new log
I20260809 15:07:16.263147 27936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:16.241956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58692 (local address 127.24.183.254:40281)
0809 15:07:16.242777 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:16.242789 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:16.242834 (+ 45us) server_negotiation.cc:407] Connection header received
0809 15:07:16.244575 (+ 1741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:16.244599 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:16.244995 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:16.245317 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:16.247048 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:16.248458 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:16.250230 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:16.250899 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:16.251029 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:16.255823 (+ 4794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:16.255840 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:16.255854 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:16.255892 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:16.259139 (+ 3247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:16.259944 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:16.259950 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:16.259956 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:16.260125 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:16.260516 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:16.260519 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:16.260521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:16.261089 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:16.261300 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:16.261760 (+ 460us) server_negotiation.cc:299] Negotiation successful
0809 15:07:16.262105 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":593,"thread_start_us":366,"threads_started":1}
I20260809 15:07:16.269089 27932 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b8c4d0b7e114003ab0f7e0a059f798f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b8c4d0b7e114003ab0f7e0a059f798f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 40281 } }
I20260809 15:07:16.269469 27932 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b8c4d0b7e114003ab0f7e0a059f798f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:16.269562 27932 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b8c4d0b7e114003ab0f7e0a059f798f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b8c4d0b7e114003ab0f7e0a059f798f, State: Initialized, Role: FOLLOWER
I20260809 15:07:16.270076 27932 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b8c4d0b7e114003ab0f7e0a059f798f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b8c4d0b7e114003ab0f7e0a059f798f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 40281 } }
I20260809 15:07:16.270294 27932 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b8c4d0b7e114003ab0f7e0a059f798f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:07:16.270426 27932 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b8c4d0b7e114003ab0f7e0a059f798f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:07:16.270573 27932 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b8c4d0b7e114003ab0f7e0a059f798f [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:16.273007 27932 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b8c4d0b7e114003ab0f7e0a059f798f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b8c4d0b7e114003ab0f7e0a059f798f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 40281 } }
I20260809 15:07:16.273553 27932 leader_election.cc:302] T 00000000000000000000000000000000 P 9b8c4d0b7e114003ab0f7e0a059f798f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b8c4d0b7e114003ab0f7e0a059f798f; no voters:
I20260809 15:07:16.274072 27932 leader_election.cc:288] T 00000000000000000000000000000000 P 9b8c4d0b7e114003ab0f7e0a059f798f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:07:16.274194 27941 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b8c4d0b7e114003ab0f7e0a059f798f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:07:16.274864 27941 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b8c4d0b7e114003ab0f7e0a059f798f [term 1 LEADER]: Becoming Leader. State: Replica: 9b8c4d0b7e114003ab0f7e0a059f798f, State: Running, Role: LEADER
I20260809 15:07:16.275393 27941 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b8c4d0b7e114003ab0f7e0a059f798f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b8c4d0b7e114003ab0f7e0a059f798f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 40281 } }
I20260809 15:07:16.276032 27932 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b8c4d0b7e114003ab0f7e0a059f798f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:07:16.278170 27944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b8c4d0b7e114003ab0f7e0a059f798f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b8c4d0b7e114003ab0f7e0a059f798f. Latest consensus state: current_term: 1 leader_uuid: "9b8c4d0b7e114003ab0f7e0a059f798f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b8c4d0b7e114003ab0f7e0a059f798f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 40281 } } }
I20260809 15:07:16.278561 27944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b8c4d0b7e114003ab0f7e0a059f798f [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:16.279028 27942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b8c4d0b7e114003ab0f7e0a059f798f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b8c4d0b7e114003ab0f7e0a059f798f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b8c4d0b7e114003ab0f7e0a059f798f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 40281 } } }
I20260809 15:07:16.279368 27942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b8c4d0b7e114003ab0f7e0a059f798f [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:16.287134 27952 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:07:16.292775 27952 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:07:16.301872 27952 catalog_manager.cc:1347] Generated new cluster ID: ab7bee4c7fb54024a8986bb3211f7c36
I20260809 15:07:16.301988 27952 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:07:16.311030 27952 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:07:16.312366 27952 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:07:16.324069 27952 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b8c4d0b7e114003ab0f7e0a059f798f: Generated new TSK 0
I20260809 15:07:16.324884 27952 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab.
Entry for principal kudu/127.24.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab.
I20260809 15:07:16.370630 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.193:0
--local_ip_for_outbound_sockets=127.24.183.193
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:40281
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:38895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:16.697760 27970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:16.698032 27970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:16.698175 27970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:16.705425 27970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:16.705530 27970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:16.705657 27970 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 15:07:16.705727 27970 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 15:07:16.705829 27970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.193
I20260809 15:07:16.715529 27970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:38895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.183.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.183.254:40281
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.193
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:16.716774 27970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:16.718142 27970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:16.730418 27982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:16.730520 27980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:16.730497 27979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:16.731582 27970 server_base.cc:1034] running on GCE node
I20260809 15:07:16.732223 27970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:16.733074 27970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:16.734241 27970 hybrid_clock.cc:648] HybridClock initialized: now 1786288036734190 us; error 45 us; skew 500 ppm
I20260809 15:07:16.734526 27970 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:16 dist-test-slave-nj21 krb5kdc[27791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288036, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:16.737578 27970 init.cc:377] Logged in from keytab as kudu/127.24.183.193@KRBTEST.COM (short username kudu)
I20260809 15:07:16.738986 27970 webserver.cc:460] Webserver started at http://127.24.183.193:34307/ using document root <none> and password file <none>
I20260809 15:07:16.739601 27970 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:16.739742 27970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:16.740077 27970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:16.742914 27970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-0/data/instance:
uuid: "98aa81f8143041c2b4d3ca9152d8786d"
format_stamp: "Formatted at 2026-08-09 15:07:16 on dist-test-slave-nj21"
server_key: "044ec8178e8de41b7f0f17897f55b520"
server_key_iv: "23b0a9b8b2d2236385687f864462a809"
server_key_version: "encryptionkey@0"
I20260809 15:07:16.743626 27970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance:
uuid: "98aa81f8143041c2b4d3ca9152d8786d"
format_stamp: "Formatted at 2026-08-09 15:07:16 on dist-test-slave-nj21"
server_key: "044ec8178e8de41b7f0f17897f55b520"
server_key_iv: "23b0a9b8b2d2236385687f864462a809"
server_key_version: "encryptionkey@0"
I20260809 15:07:16.748567 27970 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260809 15:07:16.751852 27993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:16.753087 27970 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:07:16.753221 27970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-0/wal
uuid: "98aa81f8143041c2b4d3ca9152d8786d"
format_stamp: "Formatted at 2026-08-09 15:07:16 on dist-test-slave-nj21"
server_key: "044ec8178e8de41b7f0f17897f55b520"
server_key_iv: "23b0a9b8b2d2236385687f864462a809"
server_key_version: "encryptionkey@0"
I20260809 15:07:16.753402 27970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:16.770599 27970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:16.774163 27970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:16.774420 27970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:16.777196 27970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:07:16.777354 27970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 15:07:16.777501 27970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:07:16.777607 27970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 15:07:16 dist-test-slave-nj21 krb5kdc[27791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288036, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:16.798452 28002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:16.777624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:58873 (local address 127.24.183.254:40281)
0809 15:07:16.778190 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:16.778197 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:16.779277 (+ 1080us) server_negotiation.cc:407] Connection header received
0809 15:07:16.781370 (+ 2093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:16.781375 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:16.781460 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:16.781554 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:16.783520 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:16.784721 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:16.787133 (+ 2412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:16.787828 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:16.787965 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:16.792389 (+ 4424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:16.792395 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:16.792400 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:16.792442 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:16.795011 (+ 2569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:16.795711 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:16.795716 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:16.795719 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:16.795857 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:16.796383 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:16.796389 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:16.796392 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:16.796755 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:16.796914 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:16.797841 (+ 927us) server_negotiation.cc:299] Negotiation successful
0809 15:07:16.798067 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":338,"threads_started":1}
I20260809 15:07:16.799461 28004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:16.778220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:40281 (local address 127.24.183.193:58873)
0809 15:07:16.779095 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:16.779150 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:16.781100 (+ 1950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:16.781789 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:16.781797 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:16.782455 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:16.783267 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:16.783293 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:16.784907 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:16.784912 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:16.786965 (+ 2053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:16.786981 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:16.788101 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:16.788110 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:16.788404 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:16.789063 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:16.789097 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:16.792149 (+ 3052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:16.795148 (+ 2999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:16.795157 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:16.795172 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:16.795572 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:16.795978 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:16.795982 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:16.795986 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:16.796272 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:16.797006 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:16.797012 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:16.797480 (+ 468us) client_negotiation.cc:769] Sending connection context
0809 15:07:16.797737 (+ 257us) client_negotiation.cc:240] Negotiation successful
0809 15:07:16.798313 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":341,"threads_started":1}
I20260809 15:07:16.817102 28089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:16.805272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:40281 (local address 127.24.183.193:42639)
0809 15:07:16.805844 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:16.805860 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:16.805981 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:16.806261 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:16.806266 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:16.806696 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:16.806974 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:16.806984 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:16.808006 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:16.808010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:16.809525 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:16.809541 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:16.810411 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:16.810418 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:16.810574 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:16.811114 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:16.811130 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:16.811890 (+ 760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:16.814528 (+ 2638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:16.814533 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:16.814537 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:16.814911 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:16.815251 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:16.815256 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:16.815260 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:16.815383 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:16.815889 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:16.815894 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:16.816153 (+ 259us) client_negotiation.cc:769] Sending connection context
0809 15:07:16.816344 (+ 191us) client_negotiation.cc:240] Negotiation successful
0809 15:07:16.816611 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":316,"threads_started":1}
I20260809 15:07:16.817876 28002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:16.805159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:42639 (local address 127.24.183.254:40281)
0809 15:07:16.805318 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:16.805322 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:16.805937 (+ 615us) server_negotiation.cc:407] Connection header received
0809 15:07:16.806084 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:16.806088 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:16.806148 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:16.806223 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:16.807106 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:16.807861 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:16.809663 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:16.810212 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:16.810315 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:16.812054 (+ 1739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:16.812061 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:16.812066 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:16.812098 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:16.814407 (+ 2309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:16.815017 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:16.815022 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:16.815025 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:16.815142 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:16.815481 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:16.815486 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:16.815490 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:16.815777 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:16.815915 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:16.817434 (+ 1519us) server_negotiation.cc:299] Negotiation successful
0809 15:07:16.817606 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 15:07:16.830262 27970 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.193:39591
I20260809 15:07:16.830295 28171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.193:39591 every 8 connection(s)
I20260809 15:07:16.831481 27970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
I20260809 15:07:16.833554 28172 heartbeater.cc:347] Connected to a master server at 127.24.183.254:40281
I20260809 15:07:16.833770 28172 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:16.834429 28172 heartbeater.cc:511] Master 127.24.183.254:40281 requested a full tablet report, sending...
I20260809 15:07:16.836442 27864 ts_manager.cc:194] Registered new tserver with Master: 98aa81f8143041c2b4d3ca9152d8786d (127.24.183.193:39591)
I20260809 15:07:16.837947 27864 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.193@KRBTEST.COM'} at 127.24.183.193:58873
I20260809 15:07:16.839288 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 27970
I20260809 15:07:16.839519 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance
I20260809 15:07:16.840366 25311 external_mini_cluster.cc:1428] Setting key 2e64e23da4a7ce3155253da3557f9f0a
WARNING: no policy specified for kudu/127.24.183.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab.
Entry for principal kudu/127.24.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab.
I20260809 15:07:16.880927 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.194:0
--local_ip_for_outbound_sockets=127.24.183.194
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:40281
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:38895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:17.203832 28176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:17.204118 28176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:17.204276 28176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:17.211426 28176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:17.211540 28176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:17.211694 28176 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 15:07:17.211772 28176 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 15:07:17.211879 28176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.194
I20260809 15:07:17.221489 28176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:38895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.183.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.183.254:40281
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.194
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:17.222744 28176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:17.224052 28176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:17.238250 28181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:17.238229 28182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:17.239043 28186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:17.239418 28176 server_base.cc:1034] running on GCE node
I20260809 15:07:17.240095 28176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:17.241026 28176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:17.242244 28176 hybrid_clock.cc:648] HybridClock initialized: now 1786288037242179 us; error 56 us; skew 500 ppm
I20260809 15:07:17.242543 28176 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:17 dist-test-slave-nj21 krb5kdc[27791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288037, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:17.245765 28176 init.cc:377] Logged in from keytab as kudu/127.24.183.194@KRBTEST.COM (short username kudu)
I20260809 15:07:17.247308 28176 webserver.cc:460] Webserver started at http://127.24.183.194:37143/ using document root <none> and password file <none>
I20260809 15:07:17.248057 28176 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:17.248178 28176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:17.248481 28176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:17.251356 28176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-1/data/instance:
uuid: "f13918c70fa446eb99ea081efc105f8d"
format_stamp: "Formatted at 2026-08-09 15:07:17 on dist-test-slave-nj21"
server_key: "14b745e6fb98036c3ca588c0b4f38afd"
server_key_iv: "57d8c2f731d11ddef42da177e760dbf5"
server_key_version: "encryptionkey@0"
I20260809 15:07:17.252116 28176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance:
uuid: "f13918c70fa446eb99ea081efc105f8d"
format_stamp: "Formatted at 2026-08-09 15:07:17 on dist-test-slave-nj21"
server_key: "14b745e6fb98036c3ca588c0b4f38afd"
server_key_iv: "57d8c2f731d11ddef42da177e760dbf5"
server_key_version: "encryptionkey@0"
I20260809 15:07:17.257169 28176 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260809 15:07:17.260619 28199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:17.261909 28176 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 15:07:17.262064 28176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-1/wal
uuid: "f13918c70fa446eb99ea081efc105f8d"
format_stamp: "Formatted at 2026-08-09 15:07:17 on dist-test-slave-nj21"
server_key: "14b745e6fb98036c3ca588c0b4f38afd"
server_key_iv: "57d8c2f731d11ddef42da177e760dbf5"
server_key_version: "encryptionkey@0"
I20260809 15:07:17.262241 28176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:17.279084 28176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:17.282969 28176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:17.283238 28176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:17.285941 28176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:07:17.286052 28176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:17.286170 28176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:07:17.286252 28176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 15:07:17 dist-test-slave-nj21 krb5kdc[27791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288037, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:17.309211 28002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:17.286582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:48697 (local address 127.24.183.254:40281)
0809 15:07:17.286791 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:17.286797 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:17.288206 (+ 1409us) server_negotiation.cc:407] Connection header received
0809 15:07:17.290326 (+ 2120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:17.290333 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:17.290420 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:17.290509 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:17.292958 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:17.293816 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:17.296690 (+ 2874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:17.297227 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:17.297361 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:17.302050 (+ 4689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:17.302057 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:17.302063 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:17.302106 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:17.305244 (+ 3138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:17.306190 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:17.306196 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:17.306201 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:17.306390 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:17.307056 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:17.307062 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:17.307066 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:17.307448 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:17.307615 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:17.308765 (+ 1150us) server_negotiation.cc:299] Negotiation successful
0809 15:07:17.308942 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 15:07:17.311168 28214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:17.287087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:40281 (local address 127.24.183.194:48697)
0809 15:07:17.288011 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:17.288066 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:17.290018 (+ 1952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:17.290956 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:17.290967 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:17.291807 (+ 840us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:17.292714 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:17.292740 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:17.293999 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:17.294006 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:17.296502 (+ 2496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:17.296524 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:17.297546 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:17.297558 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:17.297890 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:17.298601 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:17.298637 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:17.301794 (+ 3157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:17.305367 (+ 3573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:17.305376 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:17.305399 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:17.306034 (+ 635us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:17.306552 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:17.306561 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:17.306564 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:17.306885 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:17.307732 (+ 847us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:17.307740 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:17.308313 (+ 573us) client_negotiation.cc:769] Sending connection context
0809 15:07:17.308666 (+ 353us) client_negotiation.cc:240] Negotiation successful
0809 15:07:17.309294 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":370,"threads_started":1}
I20260809 15:07:17.331527 28329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:17.316874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:40281 (local address 127.24.183.194:39071)
0809 15:07:17.317819 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:17.317841 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:17.317980 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:17.318541 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:17.318547 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:17.319274 (+ 727us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:17.319708 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:17.319723 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:17.320755 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:17.320762 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:17.322679 (+ 1917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:17.322696 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:17.323546 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:17.323553 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:17.323756 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:17.324391 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:17.324409 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:17.325432 (+ 1023us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:17.329148 (+ 3716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:17.329151 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:17.329154 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:17.329549 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:17.329923 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:17.329926 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:17.329928 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:17.330039 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:17.330667 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:17.330670 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:17.330823 (+ 153us) client_negotiation.cc:769] Sending connection context
0809 15:07:17.330949 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 15:07:17.331180 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":775,"thread_start_us":349,"threads_started":1}
I20260809 15:07:17.331655 28002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:17.316804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:39071 (local address 127.24.183.254:40281)
0809 15:07:17.316979 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:17.316985 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:17.317921 (+ 936us) server_negotiation.cc:407] Connection header received
0809 15:07:17.318168 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:17.318173 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:17.318258 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:17.318361 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:17.319861 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:17.320618 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:17.322823 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:17.323299 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:17.323421 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:17.325646 (+ 2225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:17.325654 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:17.325659 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:17.325701 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:17.329020 (+ 3319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:17.329648 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:17.329653 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:17.329657 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:17.329823 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:17.330120 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:17.330126 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:17.330129 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:17.330568 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:17.330726 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:17.331060 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 15:07:17.331295 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 15:07:17.335565 28176 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.194:46275
I20260809 15:07:17.335610 28375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.194:46275 every 8 connection(s)
I20260809 15:07:17.336966 28176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
I20260809 15:07:17.338879 28377 heartbeater.cc:347] Connected to a master server at 127.24.183.254:40281
I20260809 15:07:17.339092 28377 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:17.339795 28377 heartbeater.cc:511] Master 127.24.183.254:40281 requested a full tablet report, sending...
I20260809 15:07:17.341302 27864 ts_manager.cc:194] Registered new tserver with Master: f13918c70fa446eb99ea081efc105f8d (127.24.183.194:46275)
I20260809 15:07:17.342106 27864 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.194@KRBTEST.COM'} at 127.24.183.194:48697
I20260809 15:07:17.346987 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 28176
I20260809 15:07:17.347159 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance
I20260809 15:07:17.347587 25311 external_mini_cluster.cc:1428] Setting key 3e9d6fccd1b22946168fa2ea9ed9a0d7
WARNING: no policy specified for kudu/127.24.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab.
Entry for principal kudu/127.24.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab.
I20260809 15:07:17.390260 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.195:0
--local_ip_for_outbound_sockets=127.24.183.195
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:40281
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:38895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:17.713716 28382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:17.713984 28382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:17.714126 28382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:17.721180 28382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:17.721283 28382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:17.721370 28382 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 15:07:17.721422 28382 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 15:07:17.721494 28382 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.195
I20260809 15:07:17.731516 28382 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:38895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.183.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.183.254:40281
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.195
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:17.732849 28382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:17.734174 28382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:17.747416 28394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:17.747443 28390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:17.747534 28388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:17.748087 28382 server_base.cc:1034] running on GCE node
I20260809 15:07:17.748699 28382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:17.749610 28382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:17.750818 28382 hybrid_clock.cc:648] HybridClock initialized: now 1786288037750763 us; error 46 us; skew 500 ppm
I20260809 15:07:17.751119 28382 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:17 dist-test-slave-nj21 krb5kdc[27791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288037, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:17.754186 28382 init.cc:377] Logged in from keytab as kudu/127.24.183.195@KRBTEST.COM (short username kudu)
I20260809 15:07:17.755565 28382 webserver.cc:460] Webserver started at http://127.24.183.195:38541/ using document root <none> and password file <none>
I20260809 15:07:17.756316 28382 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:17.756420 28382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:17.756744 28382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:17.759644 28382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-2/data/instance:
uuid: "0b6648d303b44cf39b7e8dad629f4c3a"
format_stamp: "Formatted at 2026-08-09 15:07:17 on dist-test-slave-nj21"
server_key: "bad5aeb952a3775ace3d5ff0b940f7ab"
server_key_iv: "cde2617add559909dc36e0572e294222"
server_key_version: "encryptionkey@0"
I20260809 15:07:17.760444 28382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance:
uuid: "0b6648d303b44cf39b7e8dad629f4c3a"
format_stamp: "Formatted at 2026-08-09 15:07:17 on dist-test-slave-nj21"
server_key: "bad5aeb952a3775ace3d5ff0b940f7ab"
server_key_iv: "cde2617add559909dc36e0572e294222"
server_key_version: "encryptionkey@0"
I20260809 15:07:17.765333 28382 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 15:07:17.768716 28406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:17.769923 28382 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 15:07:17.770067 28382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-2/wal
uuid: "0b6648d303b44cf39b7e8dad629f4c3a"
format_stamp: "Formatted at 2026-08-09 15:07:17 on dist-test-slave-nj21"
server_key: "bad5aeb952a3775ace3d5ff0b940f7ab"
server_key_iv: "cde2617add559909dc36e0572e294222"
server_key_version: "encryptionkey@0"
I20260809 15:07:17.770258 28382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:17.783983 28382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:17.788085 28382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:17.788347 28382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:17.790901 28382 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:07:17.791024 28382 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:17.791131 28382 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:07:17.791254 28382 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 09 15:07:17 dist-test-slave-nj21 krb5kdc[27791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288037, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:17.813856 28002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:17.791261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:34831 (local address 127.24.183.254:40281)
0809 15:07:17.791469 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:17.791476 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:17.792817 (+ 1341us) server_negotiation.cc:407] Connection header received
0809 15:07:17.794920 (+ 2103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:17.794926 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:17.795012 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:17.795102 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:17.797165 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:17.798440 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:17.801117 (+ 2677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:17.801804 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:17.801940 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:17.806364 (+ 4424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:17.806373 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:17.806379 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:17.806433 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:17.810278 (+ 3845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:17.811207 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:17.811212 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:17.811214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:17.811363 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:17.811946 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:17.811951 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:17.811955 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:17.812334 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:17.812493 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:17.813244 (+ 751us) server_negotiation.cc:299] Negotiation successful
0809 15:07:17.813502 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 15:07:17.814739 28421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:17.791733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:40281 (local address 127.24.183.195:34831)
0809 15:07:17.792630 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:17.792684 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:17.794635 (+ 1951us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:17.795327 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:17.795338 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:17.796048 (+ 710us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:17.796920 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:17.796950 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:17.798633 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:17.798640 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:17.800928 (+ 2288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:17.800947 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:17.802183 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:17.802190 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:17.802488 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:17.803228 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:17.803274 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:17.806083 (+ 2809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:17.810430 (+ 4347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:17.810437 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:17.810448 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:17.811046 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:17.811513 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:17.811518 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:17.811523 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:17.811822 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:17.812485 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:17.812493 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:17.812839 (+ 346us) client_negotiation.cc:769] Sending connection context
0809 15:07:17.813084 (+ 245us) client_negotiation.cc:240] Negotiation successful
0809 15:07:17.813560 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":375,"threads_started":1}
I20260809 15:07:17.834031 28502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:17.820403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:40281 (local address 127.24.183.195:55947)
0809 15:07:17.821277 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:17.821293 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:17.821497 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:17.821927 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:17.821930 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:17.822333 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:17.822623 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:17.822631 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:17.824108 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:17.824114 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:17.825643 (+ 1529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:17.825655 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:17.826522 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:17.826529 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:17.826708 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:17.827241 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:17.827259 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:17.828085 (+ 826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:17.831407 (+ 3322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:17.831412 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:17.831416 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:17.831863 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:17.832209 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:17.832214 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:17.832218 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:17.832340 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:17.832984 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:17.832987 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:17.833143 (+ 156us) client_negotiation.cc:769] Sending connection context
0809 15:07:17.833271 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 15:07:17.833574 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"thread_start_us":479,"threads_started":1}
I20260809 15:07:17.834077 28002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:17.820204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:55947 (local address 127.24.183.254:40281)
0809 15:07:17.820356 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:17.820360 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:17.821379 (+ 1019us) server_negotiation.cc:407] Connection header received
0809 15:07:17.821663 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:17.821670 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:17.821786 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:17.821915 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:17.822749 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:17.823965 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:17.825777 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:17.826334 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:17.826451 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:17.828308 (+ 1857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:17.828316 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:17.828322 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:17.828365 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:17.831287 (+ 2922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:17.831970 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:17.831975 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:17.831979 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:17.832109 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:17.832439 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:17.832444 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:17.832448 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:17.832788 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:17.832932 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:17.833517 (+ 585us) server_negotiation.cc:299] Negotiation successful
0809 15:07:17.833743 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 15:07:17.841228 28172 heartbeater.cc:503] Master 127.24.183.254:40281 was elected leader, sending a full tablet report...
I20260809 15:07:17.851735 28382 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.195:35827
I20260809 15:07:17.851775 28595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.195:35827 every 8 connection(s)
I20260809 15:07:17.853000 28382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
I20260809 15:07:17.854837 28597 heartbeater.cc:347] Connected to a master server at 127.24.183.254:40281
I20260809 15:07:17.855055 28597 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:17.855695 28597 heartbeater.cc:511] Master 127.24.183.254:40281 requested a full tablet report, sending...
I20260809 15:07:17.856984 27864 ts_manager.cc:194] Registered new tserver with Master: 0b6648d303b44cf39b7e8dad629f4c3a (127.24.183.195:35827)
I20260809 15:07:17.857352 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 28382
I20260809 15:07:17.857533 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance
I20260809 15:07:17.858112 25311 external_mini_cluster.cc:1428] Setting key 90ff849378895d70e41775da936add81
I20260809 15:07:17.858140 27864 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.195@KRBTEST.COM'} at 127.24.183.195:34831
I20260809 15:07:17.861078 25311 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 15:07:17 dist-test-slave-nj21 krb5kdc[27791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 15:07:17 dist-test-slave-nj21 krb5kdc[27791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:17.911494 28002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:17.897030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58700 (local address 127.24.183.254:40281)
0809 15:07:17.897334 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:17.897341 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:17.897514 (+ 173us) server_negotiation.cc:407] Connection header received
0809 15:07:17.897664 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:17.897668 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:17.897749 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:17.897849 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:17.899495 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:17.900434 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:17.901701 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:17.902178 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:17.902351 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:17.906146 (+ 3795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:17.906151 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:17.906154 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:17.906187 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:17.909003 (+ 2816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:17.909606 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:17.909611 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:17.909615 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:17.909765 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:17.910114 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:17.910119 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:17.910123 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:17.910392 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:17.910621 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:17.910918 (+ 297us) server_negotiation.cc:299] Negotiation successful
0809 15:07:17.911142 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143}
I20260809 15:07:17.915640 27864 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58700:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:07:17.922624 27864 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:07:17.977234 28617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:17.950056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37686 (local address 127.24.183.194:46275)
0809 15:07:17.955660 (+ 5604us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:17.955763 (+ 103us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:17.955871 (+ 108us) server_negotiation.cc:407] Connection header received
0809 15:07:17.957388 (+ 1517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:17.957412 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:17.958130 (+ 718us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:17.958452 (+ 322us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:07:17.965342 (+ 6890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:17.968176 (+ 2834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:17.974577 (+ 6401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:17.975958 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:17.976142 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:17.976388 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:17.976750 (+ 362us) server_negotiation.cc:299] Negotiation successful
0809 15:07:17.976935 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4878,"thread_start_us":4515,"threads_started":1}
I20260809 15:07:17.977442 28618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:17.949559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:46275 (local address 127.0.0.1:37686)
0809 15:07:17.952986 (+ 3427us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:17.953205 (+ 219us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:17.953827 (+ 622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:17.959120 (+ 5293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:17.959149 (+ 29us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:17.959320 (+ 171us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:07:17.962479 (+ 3159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:17.962517 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:17.968836 (+ 6319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:17.968854 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:17.974388 (+ 5534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:17.974404 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:17.976180 (+ 1776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:17.976187 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:17.976458 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:17.976520 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 15:07:17.976660 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 15:07:17.976914 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2505,"mutex_wait_us":2023,"thread_start_us":7111,"threads_started":1}
I20260809 15:07:17.977511 28615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:17.950271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:35827 (local address 127.0.0.1:53072)
0809 15:07:17.957595 (+ 7324us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:17.957641 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:17.957970 (+ 329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:17.960400 (+ 2430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:17.960406 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:17.960563 (+ 157us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:07:17.962244 (+ 1681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:17.962280 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:17.967400 (+ 5120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:17.967422 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:17.974582 (+ 7160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:17.974596 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:17.976656 (+ 2060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:17.976662 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:17.976848 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:17.976882 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 15:07:17.976954 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 15:07:17.977082 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6776,"mutex_wait_us":354,"thread_start_us":3049,"threads_started":1}
I20260809 15:07:17.978278 28619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:17.952332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53072 (local address 127.24.183.195:35827)
0809 15:07:17.957520 (+ 5188us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:17.957545 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:17.958196 (+ 651us) server_negotiation.cc:407] Connection header received
0809 15:07:17.958372 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:17.958383 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:17.958825 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:17.959527 (+ 702us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:07:17.962801 (+ 3274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:17.967064 (+ 4263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:17.974741 (+ 7677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:17.976198 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:17.976386 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:17.976522 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:17.977891 (+ 1369us) server_negotiation.cc:299] Negotiation successful
0809 15:07:17.978041 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4724,"thread_start_us":4337,"threads_started":1}
I20260809 15:07:17.981029 28620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:17.958088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:39591 (local address 127.0.0.1:55356)
0809 15:07:17.971421 (+ 13333us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:17.971451 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:17.971948 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:17.973673 (+ 1725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:17.973678 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:17.973709 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:07:17.974258 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:17.974272 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:17.975802 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:17.975807 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:17.977877 (+ 2070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:17.977889 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:17.980309 (+ 2420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:17.980315 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:17.980482 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:17.980514 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 15:07:17.980610 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 15:07:17.980739 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13065,"thread_start_us":12710,"threads_started":1}
I20260809 15:07:17.981114 28621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:17.958287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55356 (local address 127.24.183.193:39591)
0809 15:07:17.969598 (+ 11311us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:17.969652 (+ 54us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:17.971782 (+ 2130us) server_negotiation.cc:407] Connection header received
0809 15:07:17.972375 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:17.972404 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:17.972851 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:17.973296 (+ 445us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:07:17.974568 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:17.975625 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:17.978055 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:17.979453 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:17.979616 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:17.979861 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:17.980676 (+ 815us) server_negotiation.cc:299] Negotiation successful
0809 15:07:17.980836 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6667,"thread_start_us":6194,"threads_started":1}
I20260809 15:07:17.981690 28479 tablet_service.cc:1467] Processing CreateTablet for tablet 2835aacb5a164045b8f5e0e35211ff01 (DEFAULT_TABLE table=test-table [id=f47da7695a3247fca0969316656fb505]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:07:17.981765 28279 tablet_service.cc:1467] Processing CreateTablet for tablet 2835aacb5a164045b8f5e0e35211ff01 (DEFAULT_TABLE table=test-table [id=f47da7695a3247fca0969316656fb505]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:07:17.983011 28279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2835aacb5a164045b8f5e0e35211ff01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:17.983093 28479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2835aacb5a164045b8f5e0e35211ff01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:17.984536 28069 tablet_service.cc:1467] Processing CreateTablet for tablet 2835aacb5a164045b8f5e0e35211ff01 (DEFAULT_TABLE table=test-table [id=f47da7695a3247fca0969316656fb505]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:07:17.985836 28069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2835aacb5a164045b8f5e0e35211ff01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:17.994311 28622 tablet_bootstrap.cc:492] T 2835aacb5a164045b8f5e0e35211ff01 P f13918c70fa446eb99ea081efc105f8d: Bootstrap starting.
I20260809 15:07:17.997720 28623 tablet_bootstrap.cc:492] T 2835aacb5a164045b8f5e0e35211ff01 P 0b6648d303b44cf39b7e8dad629f4c3a: Bootstrap starting.
I20260809 15:07:17.998435 28622 tablet_bootstrap.cc:654] T 2835aacb5a164045b8f5e0e35211ff01 P f13918c70fa446eb99ea081efc105f8d: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:17.999806 28622 log.cc:824] T 2835aacb5a164045b8f5e0e35211ff01 P f13918c70fa446eb99ea081efc105f8d: Log is configured to *not* fsync() on all Append() calls
I20260809 15:07:18.001319 28623 tablet_bootstrap.cc:654] T 2835aacb5a164045b8f5e0e35211ff01 P 0b6648d303b44cf39b7e8dad629f4c3a: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:18.002526 28623 log.cc:824] T 2835aacb5a164045b8f5e0e35211ff01 P 0b6648d303b44cf39b7e8dad629f4c3a: Log is configured to *not* fsync() on all Append() calls
I20260809 15:07:18.002990 28622 tablet_bootstrap.cc:492] T 2835aacb5a164045b8f5e0e35211ff01 P f13918c70fa446eb99ea081efc105f8d: No bootstrap required, opened a new log
I20260809 15:07:18.003276 28622 ts_tablet_manager.cc:1397] T 2835aacb5a164045b8f5e0e35211ff01 P f13918c70fa446eb99ea081efc105f8d: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260809 15:07:18.003257 28624 tablet_bootstrap.cc:492] T 2835aacb5a164045b8f5e0e35211ff01 P 98aa81f8143041c2b4d3ca9152d8786d: Bootstrap starting.
I20260809 15:07:18.004879 28623 tablet_bootstrap.cc:492] T 2835aacb5a164045b8f5e0e35211ff01 P 0b6648d303b44cf39b7e8dad629f4c3a: No bootstrap required, opened a new log
I20260809 15:07:18.005115 28623 ts_tablet_manager.cc:1397] T 2835aacb5a164045b8f5e0e35211ff01 P 0b6648d303b44cf39b7e8dad629f4c3a: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 15:07:18.007231 28624 tablet_bootstrap.cc:654] T 2835aacb5a164045b8f5e0e35211ff01 P 98aa81f8143041c2b4d3ca9152d8786d: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:18.008436 28624 log.cc:824] T 2835aacb5a164045b8f5e0e35211ff01 P 98aa81f8143041c2b4d3ca9152d8786d: Log is configured to *not* fsync() on all Append() calls
I20260809 15:07:18.011243 28624 tablet_bootstrap.cc:492] T 2835aacb5a164045b8f5e0e35211ff01 P 98aa81f8143041c2b4d3ca9152d8786d: No bootstrap required, opened a new log
I20260809 15:07:18.011495 28624 ts_tablet_manager.cc:1397] T 2835aacb5a164045b8f5e0e35211ff01 P 98aa81f8143041c2b4d3ca9152d8786d: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.002s
I20260809 15:07:18.012555 28623 raft_consensus.cc:348] T 2835aacb5a164045b8f5e0e35211ff01 P 0b6648d303b44cf39b7e8dad629f4c3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } } peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } }
I20260809 15:07:18.012960 28623 raft_consensus.cc:374] T 2835aacb5a164045b8f5e0e35211ff01 P 0b6648d303b44cf39b7e8dad629f4c3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:18.013062 28623 raft_consensus.cc:729] T 2835aacb5a164045b8f5e0e35211ff01 P 0b6648d303b44cf39b7e8dad629f4c3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b6648d303b44cf39b7e8dad629f4c3a, State: Initialized, Role: FOLLOWER
I20260809 15:07:18.012981 28622 raft_consensus.cc:348] T 2835aacb5a164045b8f5e0e35211ff01 P f13918c70fa446eb99ea081efc105f8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } } peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } }
I20260809 15:07:18.013362 28622 raft_consensus.cc:374] T 2835aacb5a164045b8f5e0e35211ff01 P f13918c70fa446eb99ea081efc105f8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:18.013440 28622 raft_consensus.cc:729] T 2835aacb5a164045b8f5e0e35211ff01 P f13918c70fa446eb99ea081efc105f8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f13918c70fa446eb99ea081efc105f8d, State: Initialized, Role: FOLLOWER
I20260809 15:07:18.013640 28623 consensus_queue.cc:260] T 2835aacb5a164045b8f5e0e35211ff01 P 0b6648d303b44cf39b7e8dad629f4c3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } } peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } }
I20260809 15:07:18.014917 28597 heartbeater.cc:503] Master 127.24.183.254:40281 was elected leader, sending a full tablet report...
I20260809 15:07:18.015329 28623 ts_tablet_manager.cc:1428] T 2835aacb5a164045b8f5e0e35211ff01 P 0b6648d303b44cf39b7e8dad629f4c3a: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260809 15:07:18.015295 28622 consensus_queue.cc:260] T 2835aacb5a164045b8f5e0e35211ff01 P f13918c70fa446eb99ea081efc105f8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } } peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } }
I20260809 15:07:18.016289 28377 heartbeater.cc:503] Master 127.24.183.254:40281 was elected leader, sending a full tablet report...
I20260809 15:07:18.017158 28622 ts_tablet_manager.cc:1428] T 2835aacb5a164045b8f5e0e35211ff01 P f13918c70fa446eb99ea081efc105f8d: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260809 15:07:18.022409 28624 raft_consensus.cc:348] T 2835aacb5a164045b8f5e0e35211ff01 P 98aa81f8143041c2b4d3ca9152d8786d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } } peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } }
I20260809 15:07:18.022753 28624 raft_consensus.cc:374] T 2835aacb5a164045b8f5e0e35211ff01 P 98aa81f8143041c2b4d3ca9152d8786d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:18.022846 28624 raft_consensus.cc:729] T 2835aacb5a164045b8f5e0e35211ff01 P 98aa81f8143041c2b4d3ca9152d8786d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98aa81f8143041c2b4d3ca9152d8786d, State: Initialized, Role: FOLLOWER
I20260809 15:07:18.023337 28624 consensus_queue.cc:260] T 2835aacb5a164045b8f5e0e35211ff01 P 98aa81f8143041c2b4d3ca9152d8786d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } } peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } }
I20260809 15:07:18.024791 28624 ts_tablet_manager.cc:1428] T 2835aacb5a164045b8f5e0e35211ff01 P 98aa81f8143041c2b4d3ca9152d8786d: Time spent starting tablet: real 0.013s user 0.008s sys 0.005s
W20260809 15:07:18.083891 28173 tablet.cc:2369] T 2835aacb5a164045b8f5e0e35211ff01 P 98aa81f8143041c2b4d3ca9152d8786d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:07:18.088727 28378 tablet.cc:2369] T 2835aacb5a164045b8f5e0e35211ff01 P f13918c70fa446eb99ea081efc105f8d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:07:18.104524 28598 tablet.cc:2369] T 2835aacb5a164045b8f5e0e35211ff01 P 0b6648d303b44cf39b7e8dad629f4c3a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:07:18.236737 28629 raft_consensus.cc:482] T 2835aacb5a164045b8f5e0e35211ff01 P f13918c70fa446eb99ea081efc105f8d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:07:18.237095 28629 raft_consensus.cc:504] T 2835aacb5a164045b8f5e0e35211ff01 P f13918c70fa446eb99ea081efc105f8d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } } peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } }
I20260809 15:07:18.239171 28629 leader_election.cc:288] T 2835aacb5a164045b8f5e0e35211ff01 P f13918c70fa446eb99ea081efc105f8d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b6648d303b44cf39b7e8dad629f4c3a (127.24.183.195:35827), 98aa81f8143041c2b4d3ca9152d8786d (127.24.183.193:39591)
I20260809 15:07:18.246341 28621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:18.239950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:57149 (local address 127.24.183.193:39591)
0809 15:07:18.240169 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:18.240175 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:18.240249 (+ 74us) server_negotiation.cc:407] Connection header received
0809 15:07:18.240551 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:18.240558 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:18.240634 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:18.240691 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:07:18.241525 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:18.242322 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:18.244173 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:18.245355 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:18.245512 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:18.245587 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:18.245941 (+ 354us) server_negotiation.cc:299] Negotiation successful
0809 15:07:18.246084 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 15:07:18.247068 28632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:18.239417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:39591 (local address 127.24.183.194:57149)
0809 15:07:18.240135 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:18.240158 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:18.240382 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:18.240890 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:18.240895 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:18.240929 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:07:18.241346 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:18.241362 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:18.242456 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:18.242461 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:18.244020 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:18.244033 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:18.245561 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:18.245567 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:18.245741 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:18.245778 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 15:07:18.246510 (+ 732us) client_negotiation.cc:240] Negotiation successful
0809 15:07:18.246647 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"mutex_wait_us":23,"thread_start_us":1009,"threads_started":1}
I20260809 15:07:18.247637 28619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:18.239966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:38951 (local address 127.24.183.195:35827)
0809 15:07:18.240355 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:18.240361 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:18.240787 (+ 426us) server_negotiation.cc:407] Connection header received
0809 15:07:18.240971 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:18.240975 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:18.241040 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:18.241117 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:07:18.241791 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:18.242876 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:18.245281 (+ 2405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:18.246443 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:18.246605 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:18.246673 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:18.247306 (+ 633us) server_negotiation.cc:299] Negotiation successful
0809 15:07:18.247444 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233}
I20260809 15:07:18.248260 28631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:18.239497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:35827 (local address 127.24.183.194:38951)
0809 15:07:18.240686 (+ 1189us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:18.240700 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:18.240830 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:18.241158 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:18.241162 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:18.241191 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:07:18.241611 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:18.241624 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:18.243047 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:18.243052 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:18.245116 (+ 2064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:18.245131 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:18.246944 (+ 1813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:18.246950 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:18.247119 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:18.247151 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 15:07:18.247887 (+ 736us) client_negotiation.cc:240] Negotiation successful
0809 15:07:18.247982 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1077,"mutex_wait_us":3,"thread_start_us":555,"threads_started":1}
I20260809 15:07:18.248240 28105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2835aacb5a164045b8f5e0e35211ff01" candidate_uuid: "f13918c70fa446eb99ea081efc105f8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98aa81f8143041c2b4d3ca9152d8786d" is_pre_election: true
I20260809 15:07:18.248768 28105 raft_consensus.cc:2454] T 2835aacb5a164045b8f5e0e35211ff01 P 98aa81f8143041c2b4d3ca9152d8786d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f13918c70fa446eb99ea081efc105f8d in term 0.
I20260809 15:07:18.249187 28529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2835aacb5a164045b8f5e0e35211ff01" candidate_uuid: "f13918c70fa446eb99ea081efc105f8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" is_pre_election: true
I20260809 15:07:18.249513 28205 leader_election.cc:302] T 2835aacb5a164045b8f5e0e35211ff01 P f13918c70fa446eb99ea081efc105f8d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98aa81f8143041c2b4d3ca9152d8786d, f13918c70fa446eb99ea081efc105f8d; no voters:
I20260809 15:07:18.249760 28529 raft_consensus.cc:2454] T 2835aacb5a164045b8f5e0e35211ff01 P 0b6648d303b44cf39b7e8dad629f4c3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f13918c70fa446eb99ea081efc105f8d in term 0.
I20260809 15:07:18.249918 28629 raft_consensus.cc:2781] T 2835aacb5a164045b8f5e0e35211ff01 P f13918c70fa446eb99ea081efc105f8d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:07:18.250058 28629 raft_consensus.cc:482] T 2835aacb5a164045b8f5e0e35211ff01 P f13918c70fa446eb99ea081efc105f8d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:07:18.250154 28629 raft_consensus.cc:3037] T 2835aacb5a164045b8f5e0e35211ff01 P f13918c70fa446eb99ea081efc105f8d [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:18.252599 28629 raft_consensus.cc:504] T 2835aacb5a164045b8f5e0e35211ff01 P f13918c70fa446eb99ea081efc105f8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } } peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } }
I20260809 15:07:18.253185 28629 leader_election.cc:288] T 2835aacb5a164045b8f5e0e35211ff01 P f13918c70fa446eb99ea081efc105f8d [CANDIDATE]: Term 1 election: Requested vote from peers 0b6648d303b44cf39b7e8dad629f4c3a (127.24.183.195:35827), 98aa81f8143041c2b4d3ca9152d8786d (127.24.183.193:39591)
I20260809 15:07:18.253541 28529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2835aacb5a164045b8f5e0e35211ff01" candidate_uuid: "f13918c70fa446eb99ea081efc105f8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b6648d303b44cf39b7e8dad629f4c3a"
I20260809 15:07:18.253743 28529 raft_consensus.cc:3037] T 2835aacb5a164045b8f5e0e35211ff01 P 0b6648d303b44cf39b7e8dad629f4c3a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:18.253706 28105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2835aacb5a164045b8f5e0e35211ff01" candidate_uuid: "f13918c70fa446eb99ea081efc105f8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98aa81f8143041c2b4d3ca9152d8786d"
I20260809 15:07:18.253925 28105 raft_consensus.cc:3037] T 2835aacb5a164045b8f5e0e35211ff01 P 98aa81f8143041c2b4d3ca9152d8786d [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:18.256995 28529 raft_consensus.cc:2454] T 2835aacb5a164045b8f5e0e35211ff01 P 0b6648d303b44cf39b7e8dad629f4c3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f13918c70fa446eb99ea081efc105f8d in term 1.
I20260809 15:07:18.257088 28105 raft_consensus.cc:2454] T 2835aacb5a164045b8f5e0e35211ff01 P 98aa81f8143041c2b4d3ca9152d8786d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f13918c70fa446eb99ea081efc105f8d in term 1.
I20260809 15:07:18.257424 28200 leader_election.cc:302] T 2835aacb5a164045b8f5e0e35211ff01 P f13918c70fa446eb99ea081efc105f8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b6648d303b44cf39b7e8dad629f4c3a, f13918c70fa446eb99ea081efc105f8d; no voters:
I20260809 15:07:18.257723 28629 raft_consensus.cc:2781] T 2835aacb5a164045b8f5e0e35211ff01 P f13918c70fa446eb99ea081efc105f8d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:07:18.258118 28629 raft_consensus.cc:686] T 2835aacb5a164045b8f5e0e35211ff01 P f13918c70fa446eb99ea081efc105f8d [term 1 LEADER]: Becoming Leader. State: Replica: f13918c70fa446eb99ea081efc105f8d, State: Running, Role: LEADER
I20260809 15:07:18.258554 28629 consensus_queue.cc:237] T 2835aacb5a164045b8f5e0e35211ff01 P f13918c70fa446eb99ea081efc105f8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } } peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } }
I20260809 15:07:18.262796 27864 catalog_manager.cc:5495] T 2835aacb5a164045b8f5e0e35211ff01 P f13918c70fa446eb99ea081efc105f8d reported cstate change: term changed from 0 to 1, leader changed from <none> to f13918c70fa446eb99ea081efc105f8d (127.24.183.194). New cstate: current_term: 1 leader_uuid: "f13918c70fa446eb99ea081efc105f8d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } health_report { overall_health: UNKNOWN } } }
Aug 09 15:07:18 dist-test-slave-nj21 krb5kdc[27791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288036, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:18.359313 28647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:18.346299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:40281 (local address 127.0.0.1:58704)
0809 15:07:18.346851 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:18.346877 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:18.347008 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:18.347313 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:18.347319 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:18.348062 (+ 743us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:18.348360 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:18.348370 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:18.349410 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:18.349415 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:18.350397 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:18.350409 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:18.351136 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:18.351142 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:18.351288 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:18.351925 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:18.351942 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:18.354113 (+ 2171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:18.356962 (+ 2849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:18.356974 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:18.356982 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:18.357401 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:18.357755 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:18.357760 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:18.357762 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:18.357919 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:18.358469 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:18.358475 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:18.358634 (+ 159us) client_negotiation.cc:769] Sending connection context
0809 15:07:18.358774 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 15:07:18.358992 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":298,"threads_started":1}
I20260809 15:07:18.359400 28002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:18.346322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58704 (local address 127.24.183.254:40281)
0809 15:07:18.346487 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:18.346493 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:18.346957 (+ 464us) server_negotiation.cc:407] Connection header received
0809 15:07:18.347112 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:18.347117 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:18.347189 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:18.347270 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:18.348506 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:18.349295 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:18.350526 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:18.350962 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:18.351067 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:18.354282 (+ 3215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:18.354286 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:18.354290 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:18.354318 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:18.356835 (+ 2517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:18.357506 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:18.357512 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:18.357514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:18.357637 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:18.358008 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:18.358011 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:18.358013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:18.358305 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:18.358478 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:18.358874 (+ 396us) server_negotiation.cc:299] Negotiation successful
0809 15:07:18.359111 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":54}
I20260809 15:07:18.364351 27864 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:58704:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 15:07:18.365432 27864 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:07:18.371904 28279 tablet_service.cc:1467] Processing CreateTablet for tablet 50a573b74f77496bb5ae7915c89b9cb2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=139280ca044b42ddb6482ebaa78721b9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 15:07:18.371904 28069 tablet_service.cc:1467] Processing CreateTablet for tablet 50a573b74f77496bb5ae7915c89b9cb2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=139280ca044b42ddb6482ebaa78721b9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 15:07:18.372175 28479 tablet_service.cc:1467] Processing CreateTablet for tablet 50a573b74f77496bb5ae7915c89b9cb2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=139280ca044b42ddb6482ebaa78721b9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 15:07:18.372921 28479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50a573b74f77496bb5ae7915c89b9cb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:18.372923 28069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50a573b74f77496bb5ae7915c89b9cb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:18.372923 28279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50a573b74f77496bb5ae7915c89b9cb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:18.378474 28623 tablet_bootstrap.cc:492] T 50a573b74f77496bb5ae7915c89b9cb2 P 0b6648d303b44cf39b7e8dad629f4c3a: Bootstrap starting.
I20260809 15:07:18.379943 28623 tablet_bootstrap.cc:654] T 50a573b74f77496bb5ae7915c89b9cb2 P 0b6648d303b44cf39b7e8dad629f4c3a: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:18.380118 28624 tablet_bootstrap.cc:492] T 50a573b74f77496bb5ae7915c89b9cb2 P 98aa81f8143041c2b4d3ca9152d8786d: Bootstrap starting.
I20260809 15:07:18.381011 28622 tablet_bootstrap.cc:492] T 50a573b74f77496bb5ae7915c89b9cb2 P f13918c70fa446eb99ea081efc105f8d: Bootstrap starting.
I20260809 15:07:18.381867 28624 tablet_bootstrap.cc:654] T 50a573b74f77496bb5ae7915c89b9cb2 P 98aa81f8143041c2b4d3ca9152d8786d: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:18.382293 28622 tablet_bootstrap.cc:654] T 50a573b74f77496bb5ae7915c89b9cb2 P f13918c70fa446eb99ea081efc105f8d: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:18.384969 28623 tablet_bootstrap.cc:492] T 50a573b74f77496bb5ae7915c89b9cb2 P 0b6648d303b44cf39b7e8dad629f4c3a: No bootstrap required, opened a new log
I20260809 15:07:18.385093 28622 tablet_bootstrap.cc:492] T 50a573b74f77496bb5ae7915c89b9cb2 P f13918c70fa446eb99ea081efc105f8d: No bootstrap required, opened a new log
I20260809 15:07:18.385118 28623 ts_tablet_manager.cc:1397] T 50a573b74f77496bb5ae7915c89b9cb2 P 0b6648d303b44cf39b7e8dad629f4c3a: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.004s
I20260809 15:07:18.385231 28622 ts_tablet_manager.cc:1397] T 50a573b74f77496bb5ae7915c89b9cb2 P f13918c70fa446eb99ea081efc105f8d: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.004s
I20260809 15:07:18.385834 28624 tablet_bootstrap.cc:492] T 50a573b74f77496bb5ae7915c89b9cb2 P 98aa81f8143041c2b4d3ca9152d8786d: No bootstrap required, opened a new log
I20260809 15:07:18.385998 28624 ts_tablet_manager.cc:1397] T 50a573b74f77496bb5ae7915c89b9cb2 P 98aa81f8143041c2b4d3ca9152d8786d: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260809 15:07:18.385915 28623 raft_consensus.cc:348] T 50a573b74f77496bb5ae7915c89b9cb2 P 0b6648d303b44cf39b7e8dad629f4c3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } }
I20260809 15:07:18.385982 28622 raft_consensus.cc:348] T 50a573b74f77496bb5ae7915c89b9cb2 P f13918c70fa446eb99ea081efc105f8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } }
I20260809 15:07:18.386188 28623 raft_consensus.cc:374] T 50a573b74f77496bb5ae7915c89b9cb2 P 0b6648d303b44cf39b7e8dad629f4c3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:18.386227 28622 raft_consensus.cc:374] T 50a573b74f77496bb5ae7915c89b9cb2 P f13918c70fa446eb99ea081efc105f8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:18.386286 28623 raft_consensus.cc:729] T 50a573b74f77496bb5ae7915c89b9cb2 P 0b6648d303b44cf39b7e8dad629f4c3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b6648d303b44cf39b7e8dad629f4c3a, State: Initialized, Role: FOLLOWER
I20260809 15:07:18.386322 28622 raft_consensus.cc:729] T 50a573b74f77496bb5ae7915c89b9cb2 P f13918c70fa446eb99ea081efc105f8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f13918c70fa446eb99ea081efc105f8d, State: Initialized, Role: FOLLOWER
I20260809 15:07:18.386535 28622 consensus_queue.cc:260] T 50a573b74f77496bb5ae7915c89b9cb2 P f13918c70fa446eb99ea081efc105f8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } }
I20260809 15:07:18.386541 28623 consensus_queue.cc:260] T 50a573b74f77496bb5ae7915c89b9cb2 P 0b6648d303b44cf39b7e8dad629f4c3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } }
I20260809 15:07:18.387102 28622 ts_tablet_manager.cc:1428] T 50a573b74f77496bb5ae7915c89b9cb2 P f13918c70fa446eb99ea081efc105f8d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 15:07:18.387290 28623 ts_tablet_manager.cc:1428] T 50a573b74f77496bb5ae7915c89b9cb2 P 0b6648d303b44cf39b7e8dad629f4c3a: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260809 15:07:18.387100 28628 tablet_replica.cc:439] T 50a573b74f77496bb5ae7915c89b9cb2 P 0b6648d303b44cf39b7e8dad629f4c3a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } } }
I20260809 15:07:18.387478 28628 tablet_replica.cc:442] T 50a573b74f77496bb5ae7915c89b9cb2 P 0b6648d303b44cf39b7e8dad629f4c3a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:07:18.387806 28629 tablet_replica.cc:439] T 50a573b74f77496bb5ae7915c89b9cb2 P f13918c70fa446eb99ea081efc105f8d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } } }
I20260809 15:07:18.388067 28629 tablet_replica.cc:442] T 50a573b74f77496bb5ae7915c89b9cb2 P f13918c70fa446eb99ea081efc105f8d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:07:18.386863 28624 raft_consensus.cc:348] T 50a573b74f77496bb5ae7915c89b9cb2 P 98aa81f8143041c2b4d3ca9152d8786d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } }
I20260809 15:07:18.388968 28624 raft_consensus.cc:374] T 50a573b74f77496bb5ae7915c89b9cb2 P 98aa81f8143041c2b4d3ca9152d8786d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:18.389076 28624 raft_consensus.cc:729] T 50a573b74f77496bb5ae7915c89b9cb2 P 98aa81f8143041c2b4d3ca9152d8786d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98aa81f8143041c2b4d3ca9152d8786d, State: Initialized, Role: FOLLOWER
I20260809 15:07:18.389339 28624 consensus_queue.cc:260] T 50a573b74f77496bb5ae7915c89b9cb2 P 98aa81f8143041c2b4d3ca9152d8786d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } }
I20260809 15:07:18.390491 28624 ts_tablet_manager.cc:1428] T 50a573b74f77496bb5ae7915c89b9cb2 P 98aa81f8143041c2b4d3ca9152d8786d: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260809 15:07:18.390403 28630 tablet_replica.cc:439] T 50a573b74f77496bb5ae7915c89b9cb2 P 98aa81f8143041c2b4d3ca9152d8786d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } } }
I20260809 15:07:18.390836 28630 tablet_replica.cc:442] T 50a573b74f77496bb5ae7915c89b9cb2 P 98aa81f8143041c2b4d3ca9152d8786d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:07:18.585536 28630 raft_consensus.cc:482] T 50a573b74f77496bb5ae7915c89b9cb2 P 98aa81f8143041c2b4d3ca9152d8786d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:07:18.585791 28630 raft_consensus.cc:504] T 50a573b74f77496bb5ae7915c89b9cb2 P 98aa81f8143041c2b4d3ca9152d8786d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } }
I20260809 15:07:18.587401 28630 leader_election.cc:288] T 50a573b74f77496bb5ae7915c89b9cb2 P 98aa81f8143041c2b4d3ca9152d8786d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f13918c70fa446eb99ea081efc105f8d (127.24.183.194:46275), 0b6648d303b44cf39b7e8dad629f4c3a (127.24.183.195:35827)
I20260809 15:07:18.596493 28619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:18.587835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:41605 (local address 127.24.183.195:35827)
0809 15:07:18.588027 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:18.588035 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:18.588938 (+ 903us) server_negotiation.cc:407] Connection header received
0809 15:07:18.589081 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:18.589085 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:18.589147 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:18.589216 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:07:18.589885 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:18.591019 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:18.594075 (+ 3056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:18.595315 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:18.595470 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:18.595548 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:18.596070 (+ 522us) server_negotiation.cc:299] Negotiation successful
0809 15:07:18.596216 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 15:07:18.596544 28654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:18.587835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:42733 (local address 127.24.183.194:46275)
0809 15:07:18.589029 (+ 1194us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:18.589034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:18.589054 (+ 20us) server_negotiation.cc:407] Connection header received
0809 15:07:18.589121 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:18.589124 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:18.589182 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:18.589283 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:07:18.589994 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:18.591049 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:18.594074 (+ 3025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:18.595323 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:18.595479 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:18.595548 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:18.596080 (+ 532us) server_negotiation.cc:299] Negotiation successful
0809 15:07:18.596219 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1079,"thread_start_us":953,"threads_started":1}
I20260809 15:07:18.596662 28652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:18.587660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:35827 (local address 127.24.183.193:41605)
0809 15:07:18.588822 (+ 1162us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:18.588840 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:18.588969 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:18.589263 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:18.589268 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:18.589304 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:07:18.589713 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:18.589727 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:18.591186 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:18.591192 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:18.593917 (+ 2725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:18.593932 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:18.595580 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:18.595586 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:18.595853 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:18.595944 (+ 91us) client_negotiation.cc:769] Sending connection context
0809 15:07:18.596036 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 15:07:18.596197 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1019,"mutex_wait_us":343,"spinlock_wait_cycles":27520,"thread_start_us":419,"threads_started":1}
I20260809 15:07:18.596998 28653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:18.587652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:46275 (local address 127.24.183.193:42733)
0809 15:07:18.588482 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:18.588513 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:18.588688 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:18.589318 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:18.589323 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:18.589361 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:07:18.589813 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:18.589826 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:18.591188 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:18.591192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:18.593917 (+ 2725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:18.593932 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:18.595594 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:18.595600 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:18.595852 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:18.595943 (+ 91us) client_negotiation.cc:769] Sending connection context
0809 15:07:18.596035 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 15:07:18.596197 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"mutex_wait_us":187,"thread_start_us":853,"threads_started":1}
I20260809 15:07:18.597295 28529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50a573b74f77496bb5ae7915c89b9cb2" candidate_uuid: "98aa81f8143041c2b4d3ca9152d8786d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" is_pre_election: true
I20260809 15:07:18.597551 28529 raft_consensus.cc:2454] T 50a573b74f77496bb5ae7915c89b9cb2 P 0b6648d303b44cf39b7e8dad629f4c3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98aa81f8143041c2b4d3ca9152d8786d in term 0.
I20260809 15:07:18.597882 28303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50a573b74f77496bb5ae7915c89b9cb2" candidate_uuid: "98aa81f8143041c2b4d3ca9152d8786d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f13918c70fa446eb99ea081efc105f8d" is_pre_election: true
I20260809 15:07:18.598032 27994 leader_election.cc:302] T 50a573b74f77496bb5ae7915c89b9cb2 P 98aa81f8143041c2b4d3ca9152d8786d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b6648d303b44cf39b7e8dad629f4c3a, 98aa81f8143041c2b4d3ca9152d8786d; no voters:
I20260809 15:07:18.598213 28303 raft_consensus.cc:2454] T 50a573b74f77496bb5ae7915c89b9cb2 P f13918c70fa446eb99ea081efc105f8d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98aa81f8143041c2b4d3ca9152d8786d in term 0.
I20260809 15:07:18.598460 28630 raft_consensus.cc:2781] T 50a573b74f77496bb5ae7915c89b9cb2 P 98aa81f8143041c2b4d3ca9152d8786d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:07:18.598596 28630 raft_consensus.cc:482] T 50a573b74f77496bb5ae7915c89b9cb2 P 98aa81f8143041c2b4d3ca9152d8786d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:07:18.598704 28630 raft_consensus.cc:3037] T 50a573b74f77496bb5ae7915c89b9cb2 P 98aa81f8143041c2b4d3ca9152d8786d [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:18.601056 28630 raft_consensus.cc:504] T 50a573b74f77496bb5ae7915c89b9cb2 P 98aa81f8143041c2b4d3ca9152d8786d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } }
I20260809 15:07:18.601604 28630 leader_election.cc:288] T 50a573b74f77496bb5ae7915c89b9cb2 P 98aa81f8143041c2b4d3ca9152d8786d [CANDIDATE]: Term 1 election: Requested vote from peers f13918c70fa446eb99ea081efc105f8d (127.24.183.194:46275), 0b6648d303b44cf39b7e8dad629f4c3a (127.24.183.195:35827)
I20260809 15:07:18.602054 28529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50a573b74f77496bb5ae7915c89b9cb2" candidate_uuid: "98aa81f8143041c2b4d3ca9152d8786d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b6648d303b44cf39b7e8dad629f4c3a"
I20260809 15:07:18.602097 28303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50a573b74f77496bb5ae7915c89b9cb2" candidate_uuid: "98aa81f8143041c2b4d3ca9152d8786d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f13918c70fa446eb99ea081efc105f8d"
I20260809 15:07:18.602262 28529 raft_consensus.cc:3037] T 50a573b74f77496bb5ae7915c89b9cb2 P 0b6648d303b44cf39b7e8dad629f4c3a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:18.602288 28303 raft_consensus.cc:3037] T 50a573b74f77496bb5ae7915c89b9cb2 P f13918c70fa446eb99ea081efc105f8d [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:18.604293 28529 raft_consensus.cc:2454] T 50a573b74f77496bb5ae7915c89b9cb2 P 0b6648d303b44cf39b7e8dad629f4c3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98aa81f8143041c2b4d3ca9152d8786d in term 1.
I20260809 15:07:18.604514 28303 raft_consensus.cc:2454] T 50a573b74f77496bb5ae7915c89b9cb2 P f13918c70fa446eb99ea081efc105f8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98aa81f8143041c2b4d3ca9152d8786d in term 1.
I20260809 15:07:18.604652 27994 leader_election.cc:302] T 50a573b74f77496bb5ae7915c89b9cb2 P 98aa81f8143041c2b4d3ca9152d8786d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b6648d303b44cf39b7e8dad629f4c3a, 98aa81f8143041c2b4d3ca9152d8786d; no voters:
I20260809 15:07:18.604923 28630 raft_consensus.cc:2781] T 50a573b74f77496bb5ae7915c89b9cb2 P 98aa81f8143041c2b4d3ca9152d8786d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:07:18.605326 28630 raft_consensus.cc:686] T 50a573b74f77496bb5ae7915c89b9cb2 P 98aa81f8143041c2b4d3ca9152d8786d [term 1 LEADER]: Becoming Leader. State: Replica: 98aa81f8143041c2b4d3ca9152d8786d, State: Running, Role: LEADER
I20260809 15:07:18.605856 28630 consensus_queue.cc:237] T 50a573b74f77496bb5ae7915c89b9cb2 P 98aa81f8143041c2b4d3ca9152d8786d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } }
I20260809 15:07:18.609092 28656 tablet_replica.cc:439] T 50a573b74f77496bb5ae7915c89b9cb2 P 98aa81f8143041c2b4d3ca9152d8786d: TxnStatusTablet state changed. Reason: New leader 98aa81f8143041c2b4d3ca9152d8786d. Latest consensus state: current_term: 1 leader_uuid: "98aa81f8143041c2b4d3ca9152d8786d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } } }
I20260809 15:07:18.609530 28656 tablet_replica.cc:442] T 50a573b74f77496bb5ae7915c89b9cb2 P 98aa81f8143041c2b4d3ca9152d8786d: This TxnStatusTablet replica's current role is: LEADER
I20260809 15:07:18.610261 28659 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 15:07:18.610553 28659 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 15:07:18.610558 27864 catalog_manager.cc:5495] T 50a573b74f77496bb5ae7915c89b9cb2 P 98aa81f8143041c2b4d3ca9152d8786d reported cstate change: term changed from 0 to 1, leader changed from <none> to 98aa81f8143041c2b4d3ca9152d8786d (127.24.183.193). New cstate: current_term: 1 leader_uuid: "98aa81f8143041c2b4d3ca9152d8786d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } health_report { overall_health: UNKNOWN } } }
I20260809 15:07:18.628777 28621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:18.622779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55372 (local address 127.24.183.193:39591)
0809 15:07:18.622934 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:18.622939 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:18.622956 (+ 17us) server_negotiation.cc:407] Connection header received
0809 15:07:18.623208 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:18.623212 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:18.623301 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:18.623413 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:07:18.623834 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:18.624616 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:18.626534 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:18.627081 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:18.627190 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:18.627721 (+ 531us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:07:18.628200 (+ 479us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:07:18.628309 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:18.628455 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 15:07:18.628520 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 15:07:18.628888 28647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:18.622638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:39591 (local address 127.0.0.1:55372)
0809 15:07:18.622850 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:18.622872 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:18.623022 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:18.623465 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:18.623470 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:18.623484 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 15:07:18.623705 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:18.623717 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:18.624740 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:18.624746 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:18.626388 (+ 1642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:18.626404 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:18.627236 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:18.627242 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:18.627435 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:18.627507 (+ 72us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 15:07:18.628307 (+ 800us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 15:07:18.628338 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 15:07:18.628431 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 15:07:18.628585 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 15:07:18.637174 28303 raft_consensus.cc:1261] T 50a573b74f77496bb5ae7915c89b9cb2 P f13918c70fa446eb99ea081efc105f8d [term 1 FOLLOWER]: Refusing update from remote peer 98aa81f8143041c2b4d3ca9152d8786d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:07:18.637208 28529 raft_consensus.cc:1261] T 50a573b74f77496bb5ae7915c89b9cb2 P 0b6648d303b44cf39b7e8dad629f4c3a [term 1 FOLLOWER]: Refusing update from remote peer 98aa81f8143041c2b4d3ca9152d8786d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:07:18.638191 28656 consensus_queue.cc:1035] T 50a573b74f77496bb5ae7915c89b9cb2 P 98aa81f8143041c2b4d3ca9152d8786d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:07:18.638540 28630 consensus_queue.cc:1035] T 50a573b74f77496bb5ae7915c89b9cb2 P 98aa81f8143041c2b4d3ca9152d8786d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:07:18.651448 28628 tablet_replica.cc:439] T 50a573b74f77496bb5ae7915c89b9cb2 P 0b6648d303b44cf39b7e8dad629f4c3a: TxnStatusTablet state changed. Reason: New leader 98aa81f8143041c2b4d3ca9152d8786d. Latest consensus state: current_term: 1 leader_uuid: "98aa81f8143041c2b4d3ca9152d8786d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } } }
I20260809 15:07:18.653637 28628 tablet_replica.cc:442] T 50a573b74f77496bb5ae7915c89b9cb2 P 0b6648d303b44cf39b7e8dad629f4c3a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:07:18.653550 28629 tablet_replica.cc:439] T 50a573b74f77496bb5ae7915c89b9cb2 P f13918c70fa446eb99ea081efc105f8d: TxnStatusTablet state changed. Reason: New leader 98aa81f8143041c2b4d3ca9152d8786d. Latest consensus state: current_term: 1 leader_uuid: "98aa81f8143041c2b4d3ca9152d8786d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } } }
I20260809 15:07:18.653882 28629 tablet_replica.cc:442] T 50a573b74f77496bb5ae7915c89b9cb2 P f13918c70fa446eb99ea081efc105f8d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:07:18.656718 28663 mvcc.cc:205] Tried to move back new op lower bound from 7316635806247993344 to 7316635806133219328. Current Snapshot: MvccSnapshot[applied={T|T < 7316635806247993344}]
I20260809 15:07:18.668789 28629 tablet_replica.cc:439] T 50a573b74f77496bb5ae7915c89b9cb2 P f13918c70fa446eb99ea081efc105f8d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "98aa81f8143041c2b4d3ca9152d8786d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } } }
I20260809 15:07:18.669157 28629 tablet_replica.cc:442] T 50a573b74f77496bb5ae7915c89b9cb2 P f13918c70fa446eb99ea081efc105f8d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:07:18.670008 28630 tablet_replica.cc:439] T 50a573b74f77496bb5ae7915c89b9cb2 P 98aa81f8143041c2b4d3ca9152d8786d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "98aa81f8143041c2b4d3ca9152d8786d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } } }
I20260809 15:07:18.670948 28630 tablet_replica.cc:442] T 50a573b74f77496bb5ae7915c89b9cb2 P 98aa81f8143041c2b4d3ca9152d8786d: This TxnStatusTablet replica's current role is: LEADER
I20260809 15:07:18.674015 28630 tablet_replica.cc:439] T 50a573b74f77496bb5ae7915c89b9cb2 P 98aa81f8143041c2b4d3ca9152d8786d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "98aa81f8143041c2b4d3ca9152d8786d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } } }
I20260809 15:07:18.674254 28630 tablet_replica.cc:442] T 50a573b74f77496bb5ae7915c89b9cb2 P 98aa81f8143041c2b4d3ca9152d8786d: This TxnStatusTablet replica's current role is: LEADER
I20260809 15:07:18.675762 28664 mvcc.cc:205] Tried to move back new op lower bound from 7316635806247993344 to 7316635806133219328. Current Snapshot: MvccSnapshot[applied={T|T < 7316635806247993344}]
I20260809 15:07:18.677176 28628 tablet_replica.cc:439] T 50a573b74f77496bb5ae7915c89b9cb2 P 0b6648d303b44cf39b7e8dad629f4c3a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "98aa81f8143041c2b4d3ca9152d8786d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } } }
I20260809 15:07:18.677439 28628 tablet_replica.cc:442] T 50a573b74f77496bb5ae7915c89b9cb2 P 0b6648d303b44cf39b7e8dad629f4c3a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:07:18.686416 28654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:18.680157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37688 (local address 127.24.183.194:46275)
0809 15:07:18.680409 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:18.680416 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:18.680804 (+ 388us) server_negotiation.cc:407] Connection header received
0809 15:07:18.681028 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:18.681032 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:18.681118 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:18.681234 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:07:18.681712 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:18.682419 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:18.684327 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:18.684730 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:18.684819 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:18.685280 (+ 461us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:07:18.685833 (+ 553us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:07:18.685943 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:18.686060 (+ 117us) server_negotiation.cc:299] Negotiation successful
0809 15:07:18.686128 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 15:07:18.702859 28621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:18.696584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:43581 (local address 127.24.183.193:39591)
0809 15:07:18.696744 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:18.696749 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:18.697074 (+ 325us) server_negotiation.cc:407] Connection header received
0809 15:07:18.697254 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:18.697258 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:18.697327 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:18.697401 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:07:18.698010 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:18.698842 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:18.700888 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:18.701435 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:18.701553 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:18.702098 (+ 545us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:07:18.702256 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:07:18.702345 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:18.702486 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 15:07:18.702559 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 15:07:18.703020 28686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:18.696412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:39591 (local address 127.24.183.194:43581)
0809 15:07:18.696973 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:18.696992 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:18.697135 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:18.697430 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:18.697435 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:18.697447 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 15:07:18.697783 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:18.697793 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:18.698976 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:18.698981 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:18.700721 (+ 1740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:18.700737 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:18.701619 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:18.701626 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:18.701801 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:18.701888 (+ 87us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 15:07:18.702362 (+ 474us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 15:07:18.702375 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 15:07:18.702460 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 15:07:18.702608 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":341,"threads_started":1}
I20260809 15:07:18.713243 28529 raft_consensus.cc:1261] T 2835aacb5a164045b8f5e0e35211ff01 P 0b6648d303b44cf39b7e8dad629f4c3a [term 1 FOLLOWER]: Refusing update from remote peer f13918c70fa446eb99ea081efc105f8d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:07:18.713435 28105 raft_consensus.cc:1261] T 2835aacb5a164045b8f5e0e35211ff01 P 98aa81f8143041c2b4d3ca9152d8786d [term 1 FOLLOWER]: Refusing update from remote peer f13918c70fa446eb99ea081efc105f8d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:07:18.713893 28633 consensus_queue.cc:1035] T 2835aacb5a164045b8f5e0e35211ff01 P f13918c70fa446eb99ea081efc105f8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:07:18.714324 28629 consensus_queue.cc:1035] T 2835aacb5a164045b8f5e0e35211ff01 P f13918c70fa446eb99ea081efc105f8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:07:18.753243 28654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:18.746991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:54765 (local address 127.24.183.194:46275)
0809 15:07:18.747164 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:18.747170 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:18.747659 (+ 489us) server_negotiation.cc:407] Connection header received
0809 15:07:18.747855 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:18.747860 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:18.747943 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:18.748026 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:07:18.748758 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:18.749877 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:18.751522 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:18.751983 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:18.752091 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:18.752549 (+ 458us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:07:18.752672 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:07:18.752765 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:18.752941 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 15:07:18.753025 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 15:07:18.753500 28694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:18.746477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:46275 (local address 127.24.183.193:54765)
0809 15:07:18.747556 (+ 1079us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:18.747576 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:18.747729 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:18.748193 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:18.748196 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:18.748206 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 15:07:18.748574 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:18.748587 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:18.750051 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:18.750056 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:18.751377 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:18.751391 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:18.752165 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:18.752172 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:18.752295 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:18.752365 (+ 70us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 15:07:18.752787 (+ 422us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 15:07:18.752800 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 15:07:18.752909 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 15:07:18.753059 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"thread_start_us":698,"threads_started":1}
I20260809 15:07:18.791635 27864 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:07:18.791996 27864 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:07:18.796299 27864 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9b8c4d0b7e114003ab0f7e0a059f798f: Sending DeleteTablet for 3 replicas of tablet 2835aacb5a164045b8f5e0e35211ff01
I20260809 15:07:18.797668 28279 tablet_service.cc:1514] Processing DeleteTablet for tablet 2835aacb5a164045b8f5e0e35211ff01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:07:18 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:37686
I20260809 15:07:18.797678 28479 tablet_service.cc:1514] Processing DeleteTablet for tablet 2835aacb5a164045b8f5e0e35211ff01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:07:18 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:53072
I20260809 15:07:18.798426 28069 tablet_service.cc:1514] Processing DeleteTablet for tablet 2835aacb5a164045b8f5e0e35211ff01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:07:18 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:55356
I20260809 15:07:18.798924 27864 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58700:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:07:18.799621 27864 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:07:18.807015 28278 tablet_service.cc:1467] Processing CreateTablet for tablet 098ee90ed2bf427f92c055c2d74767a0 (DEFAULT_TABLE table=test-table [id=5195cc4044ea42048726da91211daf27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:07:18.807233 28068 tablet_service.cc:1467] Processing CreateTablet for tablet 098ee90ed2bf427f92c055c2d74767a0 (DEFAULT_TABLE table=test-table [id=5195cc4044ea42048726da91211daf27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:07:18.807582 28278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 098ee90ed2bf427f92c055c2d74767a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:18.808220 28476 tablet_service.cc:1467] Processing CreateTablet for tablet 098ee90ed2bf427f92c055c2d74767a0 (DEFAULT_TABLE table=test-table [id=5195cc4044ea42048726da91211daf27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:07:18.808728 28697 tablet_replica.cc:330] T 2835aacb5a164045b8f5e0e35211ff01 P f13918c70fa446eb99ea081efc105f8d: stopping tablet replica
I20260809 15:07:18.808761 28476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 098ee90ed2bf427f92c055c2d74767a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:18.809221 28697 raft_consensus.cc:2229] T 2835aacb5a164045b8f5e0e35211ff01 P f13918c70fa446eb99ea081efc105f8d [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:07:18.809304 28700 tablet_replica.cc:330] T 2835aacb5a164045b8f5e0e35211ff01 P 98aa81f8143041c2b4d3ca9152d8786d: stopping tablet replica
I20260809 15:07:18.809970 28697 raft_consensus.cc:2258] T 2835aacb5a164045b8f5e0e35211ff01 P f13918c70fa446eb99ea081efc105f8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:07:18.810491 28700 raft_consensus.cc:2229] T 2835aacb5a164045b8f5e0e35211ff01 P 98aa81f8143041c2b4d3ca9152d8786d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:07:18.810788 28698 tablet_replica.cc:330] T 2835aacb5a164045b8f5e0e35211ff01 P 0b6648d303b44cf39b7e8dad629f4c3a: stopping tablet replica
I20260809 15:07:18.811045 28700 raft_consensus.cc:2258] T 2835aacb5a164045b8f5e0e35211ff01 P 98aa81f8143041c2b4d3ca9152d8786d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:07:18.811183 28698 raft_consensus.cc:2229] T 2835aacb5a164045b8f5e0e35211ff01 P 0b6648d303b44cf39b7e8dad629f4c3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:07:18.811548 28698 raft_consensus.cc:2258] T 2835aacb5a164045b8f5e0e35211ff01 P 0b6648d303b44cf39b7e8dad629f4c3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:07:18.811872 28068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 098ee90ed2bf427f92c055c2d74767a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:18.813479 28698 ts_tablet_manager.cc:1905] T 2835aacb5a164045b8f5e0e35211ff01 P 0b6648d303b44cf39b7e8dad629f4c3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:07:18.813511 28700 ts_tablet_manager.cc:1905] T 2835aacb5a164045b8f5e0e35211ff01 P 98aa81f8143041c2b4d3ca9152d8786d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:07:18.819139 28622 tablet_bootstrap.cc:492] T 098ee90ed2bf427f92c055c2d74767a0 P f13918c70fa446eb99ea081efc105f8d: Bootstrap starting.
I20260809 15:07:18.820863 28622 tablet_bootstrap.cc:654] T 098ee90ed2bf427f92c055c2d74767a0 P f13918c70fa446eb99ea081efc105f8d: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:18.820883 28698 ts_tablet_manager.cc:1918] T 2835aacb5a164045b8f5e0e35211ff01 P 0b6648d303b44cf39b7e8dad629f4c3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:07:18.821087 28700 ts_tablet_manager.cc:1918] T 2835aacb5a164045b8f5e0e35211ff01 P 98aa81f8143041c2b4d3ca9152d8786d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:07:18.821197 28698 log.cc:1196] T 2835aacb5a164045b8f5e0e35211ff01 P 0b6648d303b44cf39b7e8dad629f4c3a: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-2/wal/wals/2835aacb5a164045b8f5e0e35211ff01
I20260809 15:07:18.821309 28700 log.cc:1196] T 2835aacb5a164045b8f5e0e35211ff01 P 98aa81f8143041c2b4d3ca9152d8786d: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-0/wal/wals/2835aacb5a164045b8f5e0e35211ff01
I20260809 15:07:18.821830 28700 ts_tablet_manager.cc:1939] T 2835aacb5a164045b8f5e0e35211ff01 P 98aa81f8143041c2b4d3ca9152d8786d: Deleting consensus metadata
I20260809 15:07:18.821966 28698 ts_tablet_manager.cc:1939] T 2835aacb5a164045b8f5e0e35211ff01 P 0b6648d303b44cf39b7e8dad629f4c3a: Deleting consensus metadata
I20260809 15:07:18.822448 28623 tablet_bootstrap.cc:492] T 098ee90ed2bf427f92c055c2d74767a0 P 0b6648d303b44cf39b7e8dad629f4c3a: Bootstrap starting.
I20260809 15:07:18.824170 28623 tablet_bootstrap.cc:654] T 098ee90ed2bf427f92c055c2d74767a0 P 0b6648d303b44cf39b7e8dad629f4c3a: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:18.824259 27838 catalog_manager.cc:4841] TS 0b6648d303b44cf39b7e8dad629f4c3a (127.24.183.195:35827): tablet 2835aacb5a164045b8f5e0e35211ff01 (table test-table [id=f47da7695a3247fca0969316656fb505]) successfully deleted
I20260809 15:07:18.825193 28624 tablet_bootstrap.cc:492] T 098ee90ed2bf427f92c055c2d74767a0 P 98aa81f8143041c2b4d3ca9152d8786d: Bootstrap starting.
I20260809 15:07:18.826948 28624 tablet_bootstrap.cc:654] T 098ee90ed2bf427f92c055c2d74767a0 P 98aa81f8143041c2b4d3ca9152d8786d: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:18.829788 27840 catalog_manager.cc:4841] TS 98aa81f8143041c2b4d3ca9152d8786d (127.24.183.193:39591): tablet 2835aacb5a164045b8f5e0e35211ff01 (table test-table [id=f47da7695a3247fca0969316656fb505]) successfully deleted
I20260809 15:07:18.831079 28623 tablet_bootstrap.cc:492] T 098ee90ed2bf427f92c055c2d74767a0 P 0b6648d303b44cf39b7e8dad629f4c3a: No bootstrap required, opened a new log
I20260809 15:07:18.831248 28623 ts_tablet_manager.cc:1397] T 098ee90ed2bf427f92c055c2d74767a0 P 0b6648d303b44cf39b7e8dad629f4c3a: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20260809 15:07:18.831501 28697 ts_tablet_manager.cc:1905] T 2835aacb5a164045b8f5e0e35211ff01 P f13918c70fa446eb99ea081efc105f8d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:07:18.832140 28624 tablet_bootstrap.cc:492] T 098ee90ed2bf427f92c055c2d74767a0 P 98aa81f8143041c2b4d3ca9152d8786d: No bootstrap required, opened a new log
I20260809 15:07:18.832083 28623 raft_consensus.cc:348] T 098ee90ed2bf427f92c055c2d74767a0 P 0b6648d303b44cf39b7e8dad629f4c3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } }
I20260809 15:07:18.832296 28624 ts_tablet_manager.cc:1397] T 098ee90ed2bf427f92c055c2d74767a0 P 98aa81f8143041c2b4d3ca9152d8786d: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260809 15:07:18.832329 28623 raft_consensus.cc:374] T 098ee90ed2bf427f92c055c2d74767a0 P 0b6648d303b44cf39b7e8dad629f4c3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:18.832401 28623 raft_consensus.cc:729] T 098ee90ed2bf427f92c055c2d74767a0 P 0b6648d303b44cf39b7e8dad629f4c3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b6648d303b44cf39b7e8dad629f4c3a, State: Initialized, Role: FOLLOWER
I20260809 15:07:18.832680 28623 consensus_queue.cc:260] T 098ee90ed2bf427f92c055c2d74767a0 P 0b6648d303b44cf39b7e8dad629f4c3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } }
I20260809 15:07:18.832940 28624 raft_consensus.cc:348] T 098ee90ed2bf427f92c055c2d74767a0 P 98aa81f8143041c2b4d3ca9152d8786d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } }
I20260809 15:07:18.833168 28624 raft_consensus.cc:374] T 098ee90ed2bf427f92c055c2d74767a0 P 98aa81f8143041c2b4d3ca9152d8786d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:18.833232 28624 raft_consensus.cc:729] T 098ee90ed2bf427f92c055c2d74767a0 P 98aa81f8143041c2b4d3ca9152d8786d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98aa81f8143041c2b4d3ca9152d8786d, State: Initialized, Role: FOLLOWER
I20260809 15:07:18.833678 28624 consensus_queue.cc:260] T 098ee90ed2bf427f92c055c2d74767a0 P 98aa81f8143041c2b4d3ca9152d8786d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } }
I20260809 15:07:18.836310 28623 ts_tablet_manager.cc:1428] T 098ee90ed2bf427f92c055c2d74767a0 P 0b6648d303b44cf39b7e8dad629f4c3a: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260809 15:07:18.836763 28624 ts_tablet_manager.cc:1428] T 098ee90ed2bf427f92c055c2d74767a0 P 98aa81f8143041c2b4d3ca9152d8786d: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260809 15:07:18.837949 28622 tablet_bootstrap.cc:492] T 098ee90ed2bf427f92c055c2d74767a0 P f13918c70fa446eb99ea081efc105f8d: No bootstrap required, opened a new log
I20260809 15:07:18.838132 28622 ts_tablet_manager.cc:1397] T 098ee90ed2bf427f92c055c2d74767a0 P f13918c70fa446eb99ea081efc105f8d: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.000s
I20260809 15:07:18.838878 28622 raft_consensus.cc:348] T 098ee90ed2bf427f92c055c2d74767a0 P f13918c70fa446eb99ea081efc105f8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } }
I20260809 15:07:18.839121 28622 raft_consensus.cc:374] T 098ee90ed2bf427f92c055c2d74767a0 P f13918c70fa446eb99ea081efc105f8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:18.839192 28622 raft_consensus.cc:729] T 098ee90ed2bf427f92c055c2d74767a0 P f13918c70fa446eb99ea081efc105f8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f13918c70fa446eb99ea081efc105f8d, State: Initialized, Role: FOLLOWER
I20260809 15:07:18.839424 28622 consensus_queue.cc:260] T 098ee90ed2bf427f92c055c2d74767a0 P f13918c70fa446eb99ea081efc105f8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } }
I20260809 15:07:18.839591 28697 ts_tablet_manager.cc:1918] T 2835aacb5a164045b8f5e0e35211ff01 P f13918c70fa446eb99ea081efc105f8d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:07:18.840060 28622 ts_tablet_manager.cc:1428] T 098ee90ed2bf427f92c055c2d74767a0 P f13918c70fa446eb99ea081efc105f8d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 15:07:18.848034 28697 log.cc:1196] T 2835aacb5a164045b8f5e0e35211ff01 P f13918c70fa446eb99ea081efc105f8d: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-1/wal/wals/2835aacb5a164045b8f5e0e35211ff01
I20260809 15:07:18.848706 28697 ts_tablet_manager.cc:1939] T 2835aacb5a164045b8f5e0e35211ff01 P f13918c70fa446eb99ea081efc105f8d: Deleting consensus metadata
I20260809 15:07:18.850452 27840 catalog_manager.cc:4841] TS f13918c70fa446eb99ea081efc105f8d (127.24.183.194:46275): tablet 2835aacb5a164045b8f5e0e35211ff01 (table test-table [id=f47da7695a3247fca0969316656fb505]) successfully deleted
I20260809 15:07:18.964326 28630 raft_consensus.cc:482] T 098ee90ed2bf427f92c055c2d74767a0 P 98aa81f8143041c2b4d3ca9152d8786d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:07:18.964579 28630 raft_consensus.cc:504] T 098ee90ed2bf427f92c055c2d74767a0 P 98aa81f8143041c2b4d3ca9152d8786d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } }
I20260809 15:07:18.965415 28630 leader_election.cc:288] T 098ee90ed2bf427f92c055c2d74767a0 P 98aa81f8143041c2b4d3ca9152d8786d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f13918c70fa446eb99ea081efc105f8d (127.24.183.194:46275), 0b6648d303b44cf39b7e8dad629f4c3a (127.24.183.195:35827)
I20260809 15:07:18.966363 28529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "098ee90ed2bf427f92c055c2d74767a0" candidate_uuid: "98aa81f8143041c2b4d3ca9152d8786d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" is_pre_election: true
I20260809 15:07:18.966634 28529 raft_consensus.cc:2454] T 098ee90ed2bf427f92c055c2d74767a0 P 0b6648d303b44cf39b7e8dad629f4c3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98aa81f8143041c2b4d3ca9152d8786d in term 0.
I20260809 15:07:18.966725 28303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "098ee90ed2bf427f92c055c2d74767a0" candidate_uuid: "98aa81f8143041c2b4d3ca9152d8786d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f13918c70fa446eb99ea081efc105f8d" is_pre_election: true
I20260809 15:07:18.967054 28303 raft_consensus.cc:2454] T 098ee90ed2bf427f92c055c2d74767a0 P f13918c70fa446eb99ea081efc105f8d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98aa81f8143041c2b4d3ca9152d8786d in term 0.
I20260809 15:07:18.967602 27994 leader_election.cc:302] T 098ee90ed2bf427f92c055c2d74767a0 P 98aa81f8143041c2b4d3ca9152d8786d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b6648d303b44cf39b7e8dad629f4c3a, 98aa81f8143041c2b4d3ca9152d8786d; no voters:
I20260809 15:07:18.967998 28630 raft_consensus.cc:2781] T 098ee90ed2bf427f92c055c2d74767a0 P 98aa81f8143041c2b4d3ca9152d8786d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:07:18.968133 28630 raft_consensus.cc:482] T 098ee90ed2bf427f92c055c2d74767a0 P 98aa81f8143041c2b4d3ca9152d8786d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:07:18.968232 28630 raft_consensus.cc:3037] T 098ee90ed2bf427f92c055c2d74767a0 P 98aa81f8143041c2b4d3ca9152d8786d [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:18.971500 28630 raft_consensus.cc:504] T 098ee90ed2bf427f92c055c2d74767a0 P 98aa81f8143041c2b4d3ca9152d8786d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } }
I20260809 15:07:18.972692 28303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "098ee90ed2bf427f92c055c2d74767a0" candidate_uuid: "98aa81f8143041c2b4d3ca9152d8786d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f13918c70fa446eb99ea081efc105f8d"
I20260809 15:07:18.972929 28303 raft_consensus.cc:3037] T 098ee90ed2bf427f92c055c2d74767a0 P f13918c70fa446eb99ea081efc105f8d [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:18.974172 28529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "098ee90ed2bf427f92c055c2d74767a0" candidate_uuid: "98aa81f8143041c2b4d3ca9152d8786d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b6648d303b44cf39b7e8dad629f4c3a"
I20260809 15:07:18.974411 28529 raft_consensus.cc:3037] T 098ee90ed2bf427f92c055c2d74767a0 P 0b6648d303b44cf39b7e8dad629f4c3a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:18.976394 28303 raft_consensus.cc:2454] T 098ee90ed2bf427f92c055c2d74767a0 P f13918c70fa446eb99ea081efc105f8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98aa81f8143041c2b4d3ca9152d8786d in term 1.
I20260809 15:07:18.976902 27996 leader_election.cc:302] T 098ee90ed2bf427f92c055c2d74767a0 P 98aa81f8143041c2b4d3ca9152d8786d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98aa81f8143041c2b4d3ca9152d8786d, f13918c70fa446eb99ea081efc105f8d; no voters:
I20260809 15:07:18.977257 28670 raft_consensus.cc:2781] T 098ee90ed2bf427f92c055c2d74767a0 P 98aa81f8143041c2b4d3ca9152d8786d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:07:18.977460 28670 raft_consensus.cc:686] T 098ee90ed2bf427f92c055c2d74767a0 P 98aa81f8143041c2b4d3ca9152d8786d [term 1 LEADER]: Becoming Leader. State: Replica: 98aa81f8143041c2b4d3ca9152d8786d, State: Running, Role: LEADER
I20260809 15:07:18.977510 28529 raft_consensus.cc:2454] T 098ee90ed2bf427f92c055c2d74767a0 P 0b6648d303b44cf39b7e8dad629f4c3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98aa81f8143041c2b4d3ca9152d8786d in term 1.
I20260809 15:07:18.977810 28670 consensus_queue.cc:237] T 098ee90ed2bf427f92c055c2d74767a0 P 98aa81f8143041c2b4d3ca9152d8786d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } }
I20260809 15:07:18.978289 28630 leader_election.cc:288] T 098ee90ed2bf427f92c055c2d74767a0 P 98aa81f8143041c2b4d3ca9152d8786d [CANDIDATE]: Term 1 election: Requested vote from peers f13918c70fa446eb99ea081efc105f8d (127.24.183.194:46275), 0b6648d303b44cf39b7e8dad629f4c3a (127.24.183.195:35827)
I20260809 15:07:18.981104 27863 catalog_manager.cc:5495] T 098ee90ed2bf427f92c055c2d74767a0 P 98aa81f8143041c2b4d3ca9152d8786d reported cstate change: term changed from 0 to 1, leader changed from <none> to 98aa81f8143041c2b4d3ca9152d8786d (127.24.183.193). New cstate: current_term: 1 leader_uuid: "98aa81f8143041c2b4d3ca9152d8786d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } health_report { overall_health: UNKNOWN } } }
I20260809 15:07:19.033044 28621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:19.026658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55384 (local address 127.24.183.193:39591)
0809 15:07:19.026860 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:19.026866 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:19.026885 (+ 19us) server_negotiation.cc:407] Connection header received
0809 15:07:19.027027 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:19.027033 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:19.027123 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:19.027223 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:07:19.027660 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:19.028901 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:19.030806 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:19.031513 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:19.031690 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:19.032174 (+ 484us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:07:19.032339 (+ 165us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:07:19.032427 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:19.032601 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 15:07:19.032690 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 15:07:19.048643 28621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:19.042617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:44101 (local address 127.24.183.193:39591)
0809 15:07:19.042787 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:19.042793 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:19.042947 (+ 154us) server_negotiation.cc:407] Connection header received
0809 15:07:19.043102 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:19.043111 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:19.043204 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:19.043287 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:07:19.043697 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:19.044845 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:19.046689 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:19.047311 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:19.047430 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:19.047873 (+ 443us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:07:19.048009 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:07:19.048097 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:19.048215 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 15:07:19.048293 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 15:07:19.048646 28694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:19.042576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:39591 (local address 127.24.183.193:44101)
0809 15:07:19.042852 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:19.042877 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:19.043000 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:19.043326 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:19.043331 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:19.043343 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 15:07:19.043543 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:19.043553 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:19.045001 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:19.045006 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:19.046543 (+ 1537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:19.046557 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:19.047466 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:19.047472 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:19.047646 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:19.047719 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 15:07:19.048106 (+ 387us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 15:07:19.048117 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 15:07:19.048187 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 15:07:19.048327 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129}
I20260809 15:07:19.062414 28303 raft_consensus.cc:1261] T 098ee90ed2bf427f92c055c2d74767a0 P f13918c70fa446eb99ea081efc105f8d [term 1 FOLLOWER]: Refusing update from remote peer 98aa81f8143041c2b4d3ca9152d8786d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:07:19.062438 28529 raft_consensus.cc:1261] T 098ee90ed2bf427f92c055c2d74767a0 P 0b6648d303b44cf39b7e8dad629f4c3a [term 1 FOLLOWER]: Refusing update from remote peer 98aa81f8143041c2b4d3ca9152d8786d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:07:19.063141 28670 consensus_queue.cc:1035] T 098ee90ed2bf427f92c055c2d74767a0 P 98aa81f8143041c2b4d3ca9152d8786d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:07:19.063604 28670 consensus_queue.cc:1035] T 098ee90ed2bf427f92c055c2d74767a0 P 98aa81f8143041c2b4d3ca9152d8786d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 15:07:19.103353 28663 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 15:07:20.705083 27863 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:07:20.705338 27863 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:07:20.709196 27863 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9b8c4d0b7e114003ab0f7e0a059f798f: Sending DeleteTablet for 3 replicas of tablet 098ee90ed2bf427f92c055c2d74767a0
I20260809 15:07:20.709889 28068 tablet_service.cc:1514] Processing DeleteTablet for tablet 098ee90ed2bf427f92c055c2d74767a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:07:20 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:55356
I20260809 15:07:20.709965 28476 tablet_service.cc:1514] Processing DeleteTablet for tablet 098ee90ed2bf427f92c055c2d74767a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:07:20 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:53072
I20260809 15:07:20.709986 28278 tablet_service.cc:1514] Processing DeleteTablet for tablet 098ee90ed2bf427f92c055c2d74767a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:07:20 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:37686
I20260809 15:07:20.710564 28747 tablet_replica.cc:330] T 098ee90ed2bf427f92c055c2d74767a0 P 98aa81f8143041c2b4d3ca9152d8786d: stopping tablet replica
I20260809 15:07:20.710759 28752 tablet_replica.cc:330] T 098ee90ed2bf427f92c055c2d74767a0 P 0b6648d303b44cf39b7e8dad629f4c3a: stopping tablet replica
I20260809 15:07:20.710813 28747 raft_consensus.cc:2229] T 098ee90ed2bf427f92c055c2d74767a0 P 98aa81f8143041c2b4d3ca9152d8786d [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:07:20.710976 28752 raft_consensus.cc:2229] T 098ee90ed2bf427f92c055c2d74767a0 P 0b6648d303b44cf39b7e8dad629f4c3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:07:20.711222 28752 raft_consensus.cc:2258] T 098ee90ed2bf427f92c055c2d74767a0 P 0b6648d303b44cf39b7e8dad629f4c3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:07:20.711741 28754 tablet_replica.cc:330] T 098ee90ed2bf427f92c055c2d74767a0 P f13918c70fa446eb99ea081efc105f8d: stopping tablet replica
I20260809 15:07:20.711931 28754 raft_consensus.cc:2229] T 098ee90ed2bf427f92c055c2d74767a0 P f13918c70fa446eb99ea081efc105f8d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:07:20.712138 28754 raft_consensus.cc:2258] T 098ee90ed2bf427f92c055c2d74767a0 P f13918c70fa446eb99ea081efc105f8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:07:20.712227 28752 ts_tablet_manager.cc:1905] T 098ee90ed2bf427f92c055c2d74767a0 P 0b6648d303b44cf39b7e8dad629f4c3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:07:20.713021 28754 ts_tablet_manager.cc:1905] T 098ee90ed2bf427f92c055c2d74767a0 P f13918c70fa446eb99ea081efc105f8d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:07:20.714493 28747 raft_consensus.cc:2258] T 098ee90ed2bf427f92c055c2d74767a0 P 98aa81f8143041c2b4d3ca9152d8786d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:07:20.715473 28747 ts_tablet_manager.cc:1905] T 098ee90ed2bf427f92c055c2d74767a0 P 98aa81f8143041c2b4d3ca9152d8786d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:07:20.718799 28754 ts_tablet_manager.cc:1918] T 098ee90ed2bf427f92c055c2d74767a0 P f13918c70fa446eb99ea081efc105f8d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 15:07:20.718989 28754 log.cc:1196] T 098ee90ed2bf427f92c055c2d74767a0 P f13918c70fa446eb99ea081efc105f8d: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-1/wal/wals/098ee90ed2bf427f92c055c2d74767a0
I20260809 15:07:20.719498 28754 ts_tablet_manager.cc:1939] T 098ee90ed2bf427f92c055c2d74767a0 P f13918c70fa446eb99ea081efc105f8d: Deleting consensus metadata
I20260809 15:07:20.720611 27840 catalog_manager.cc:4841] TS f13918c70fa446eb99ea081efc105f8d (127.24.183.194:46275): tablet 098ee90ed2bf427f92c055c2d74767a0 (table test-table [id=5195cc4044ea42048726da91211daf27]) successfully deleted
I20260809 15:07:20.721807 28752 ts_tablet_manager.cc:1918] T 098ee90ed2bf427f92c055c2d74767a0 P 0b6648d303b44cf39b7e8dad629f4c3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 15:07:20.721979 28752 log.cc:1196] T 098ee90ed2bf427f92c055c2d74767a0 P 0b6648d303b44cf39b7e8dad629f4c3a: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-2/wal/wals/098ee90ed2bf427f92c055c2d74767a0
I20260809 15:07:20.722347 28752 ts_tablet_manager.cc:1939] T 098ee90ed2bf427f92c055c2d74767a0 P 0b6648d303b44cf39b7e8dad629f4c3a: Deleting consensus metadata
I20260809 15:07:20.723196 27838 catalog_manager.cc:4841] TS 0b6648d303b44cf39b7e8dad629f4c3a (127.24.183.195:35827): tablet 098ee90ed2bf427f92c055c2d74767a0 (table test-table [id=5195cc4044ea42048726da91211daf27]) successfully deleted
I20260809 15:07:20.723500 28747 ts_tablet_manager.cc:1918] T 098ee90ed2bf427f92c055c2d74767a0 P 98aa81f8143041c2b4d3ca9152d8786d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 15:07:20.723608 28747 log.cc:1196] T 098ee90ed2bf427f92c055c2d74767a0 P 98aa81f8143041c2b4d3ca9152d8786d: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-0/wal/wals/098ee90ed2bf427f92c055c2d74767a0
I20260809 15:07:20.724013 28747 ts_tablet_manager.cc:1939] T 098ee90ed2bf427f92c055c2d74767a0 P 98aa81f8143041c2b4d3ca9152d8786d: Deleting consensus metadata
I20260809 15:07:20.724830 27840 catalog_manager.cc:4841] TS 98aa81f8143041c2b4d3ca9152d8786d (127.24.183.193:39591): tablet 098ee90ed2bf427f92c055c2d74767a0 (table test-table [id=5195cc4044ea42048726da91211daf27]) successfully deleted
Aug 09 15:07:20 dist-test-slave-nj21 krb5kdc[27791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288040, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 15:07:20 dist-test-slave-nj21 krb5kdc[27791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288040, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:20.767021 28765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:20.752214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58708 (local address 127.24.183.254:40281)
0809 15:07:20.752812 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:20.752820 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:20.752871 (+ 51us) server_negotiation.cc:407] Connection header received
0809 15:07:20.753104 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:20.753109 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:20.753183 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:20.753310 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:20.754790 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:20.755631 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:20.757075 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:20.757508 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:20.757657 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:20.761720 (+ 4063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:20.761733 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:20.761737 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:20.761775 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:20.764428 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:20.765127 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:20.765132 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:20.765134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:20.765273 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:20.765654 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:20.765660 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:20.765663 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:20.766000 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:20.766244 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:20.766491 (+ 247us) server_negotiation.cc:299] Negotiation successful
0809 15:07:20.766661 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":268,"threads_started":1}
I20260809 15:07:20.770439 27863 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58708:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:07:20.771047 27863 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:07:20.777865 28068 tablet_service.cc:1467] Processing CreateTablet for tablet de89bfb4cdeb49639de6b2d94bb1c70b (DEFAULT_TABLE table=test-table [id=626b5b89ef494af4ac99b7ba089268d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:07:20.777928 28278 tablet_service.cc:1467] Processing CreateTablet for tablet de89bfb4cdeb49639de6b2d94bb1c70b (DEFAULT_TABLE table=test-table [id=626b5b89ef494af4ac99b7ba089268d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:07:20.778332 28068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de89bfb4cdeb49639de6b2d94bb1c70b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:20.778446 28278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de89bfb4cdeb49639de6b2d94bb1c70b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:20.781014 28476 tablet_service.cc:1467] Processing CreateTablet for tablet de89bfb4cdeb49639de6b2d94bb1c70b (DEFAULT_TABLE table=test-table [id=626b5b89ef494af4ac99b7ba089268d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:07:20.781456 28476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de89bfb4cdeb49639de6b2d94bb1c70b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:20.784967 28767 tablet_bootstrap.cc:492] T de89bfb4cdeb49639de6b2d94bb1c70b P 98aa81f8143041c2b4d3ca9152d8786d: Bootstrap starting.
I20260809 15:07:20.787607 28768 tablet_bootstrap.cc:492] T de89bfb4cdeb49639de6b2d94bb1c70b P 0b6648d303b44cf39b7e8dad629f4c3a: Bootstrap starting.
I20260809 15:07:20.788245 28766 tablet_bootstrap.cc:492] T de89bfb4cdeb49639de6b2d94bb1c70b P f13918c70fa446eb99ea081efc105f8d: Bootstrap starting.
I20260809 15:07:20.788326 28767 tablet_bootstrap.cc:654] T de89bfb4cdeb49639de6b2d94bb1c70b P 98aa81f8143041c2b4d3ca9152d8786d: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:20.789330 28768 tablet_bootstrap.cc:654] T de89bfb4cdeb49639de6b2d94bb1c70b P 0b6648d303b44cf39b7e8dad629f4c3a: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:20.790019 28766 tablet_bootstrap.cc:654] T de89bfb4cdeb49639de6b2d94bb1c70b P f13918c70fa446eb99ea081efc105f8d: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:20.790560 28767 tablet_bootstrap.cc:492] T de89bfb4cdeb49639de6b2d94bb1c70b P 98aa81f8143041c2b4d3ca9152d8786d: No bootstrap required, opened a new log
I20260809 15:07:20.790724 28767 ts_tablet_manager.cc:1397] T de89bfb4cdeb49639de6b2d94bb1c70b P 98aa81f8143041c2b4d3ca9152d8786d: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260809 15:07:20.791656 28768 tablet_bootstrap.cc:492] T de89bfb4cdeb49639de6b2d94bb1c70b P 0b6648d303b44cf39b7e8dad629f4c3a: No bootstrap required, opened a new log
I20260809 15:07:20.791842 28768 ts_tablet_manager.cc:1397] T de89bfb4cdeb49639de6b2d94bb1c70b P 0b6648d303b44cf39b7e8dad629f4c3a: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.001s
I20260809 15:07:20.792321 28767 raft_consensus.cc:348] T de89bfb4cdeb49639de6b2d94bb1c70b P 98aa81f8143041c2b4d3ca9152d8786d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } }
I20260809 15:07:20.792557 28767 raft_consensus.cc:374] T de89bfb4cdeb49639de6b2d94bb1c70b P 98aa81f8143041c2b4d3ca9152d8786d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:20.792678 28767 raft_consensus.cc:729] T de89bfb4cdeb49639de6b2d94bb1c70b P 98aa81f8143041c2b4d3ca9152d8786d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98aa81f8143041c2b4d3ca9152d8786d, State: Initialized, Role: FOLLOWER
I20260809 15:07:20.792634 28768 raft_consensus.cc:348] T de89bfb4cdeb49639de6b2d94bb1c70b P 0b6648d303b44cf39b7e8dad629f4c3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } }
I20260809 15:07:20.792904 28768 raft_consensus.cc:374] T de89bfb4cdeb49639de6b2d94bb1c70b P 0b6648d303b44cf39b7e8dad629f4c3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:20.793011 28768 raft_consensus.cc:729] T de89bfb4cdeb49639de6b2d94bb1c70b P 0b6648d303b44cf39b7e8dad629f4c3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b6648d303b44cf39b7e8dad629f4c3a, State: Initialized, Role: FOLLOWER
I20260809 15:07:20.792946 28767 consensus_queue.cc:260] T de89bfb4cdeb49639de6b2d94bb1c70b P 98aa81f8143041c2b4d3ca9152d8786d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } }
I20260809 15:07:20.793274 28768 consensus_queue.cc:260] T de89bfb4cdeb49639de6b2d94bb1c70b P 0b6648d303b44cf39b7e8dad629f4c3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } }
I20260809 15:07:20.793617 28767 ts_tablet_manager.cc:1428] T de89bfb4cdeb49639de6b2d94bb1c70b P 98aa81f8143041c2b4d3ca9152d8786d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 15:07:20.794227 28768 ts_tablet_manager.cc:1428] T de89bfb4cdeb49639de6b2d94bb1c70b P 0b6648d303b44cf39b7e8dad629f4c3a: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 15:07:20.799556 28766 tablet_bootstrap.cc:492] T de89bfb4cdeb49639de6b2d94bb1c70b P f13918c70fa446eb99ea081efc105f8d: No bootstrap required, opened a new log
I20260809 15:07:20.799722 28766 ts_tablet_manager.cc:1397] T de89bfb4cdeb49639de6b2d94bb1c70b P f13918c70fa446eb99ea081efc105f8d: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260809 15:07:20.800365 28766 raft_consensus.cc:348] T de89bfb4cdeb49639de6b2d94bb1c70b P f13918c70fa446eb99ea081efc105f8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } }
I20260809 15:07:20.800566 28766 raft_consensus.cc:374] T de89bfb4cdeb49639de6b2d94bb1c70b P f13918c70fa446eb99ea081efc105f8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:20.800657 28766 raft_consensus.cc:729] T de89bfb4cdeb49639de6b2d94bb1c70b P f13918c70fa446eb99ea081efc105f8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f13918c70fa446eb99ea081efc105f8d, State: Initialized, Role: FOLLOWER
I20260809 15:07:20.800879 28766 consensus_queue.cc:260] T de89bfb4cdeb49639de6b2d94bb1c70b P f13918c70fa446eb99ea081efc105f8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } }
I20260809 15:07:20.801651 28766 ts_tablet_manager.cc:1428] T de89bfb4cdeb49639de6b2d94bb1c70b P f13918c70fa446eb99ea081efc105f8d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 15:07:20.973546 28778 raft_consensus.cc:482] T de89bfb4cdeb49639de6b2d94bb1c70b P f13918c70fa446eb99ea081efc105f8d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:07:20.973821 28778 raft_consensus.cc:504] T de89bfb4cdeb49639de6b2d94bb1c70b P f13918c70fa446eb99ea081efc105f8d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } }
I20260809 15:07:20.974555 28778 leader_election.cc:288] T de89bfb4cdeb49639de6b2d94bb1c70b P f13918c70fa446eb99ea081efc105f8d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98aa81f8143041c2b4d3ca9152d8786d (127.24.183.193:39591), 0b6648d303b44cf39b7e8dad629f4c3a (127.24.183.195:35827)
I20260809 15:07:20.975168 28529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de89bfb4cdeb49639de6b2d94bb1c70b" candidate_uuid: "f13918c70fa446eb99ea081efc105f8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" is_pre_election: true
I20260809 15:07:20.975167 28105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de89bfb4cdeb49639de6b2d94bb1c70b" candidate_uuid: "f13918c70fa446eb99ea081efc105f8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98aa81f8143041c2b4d3ca9152d8786d" is_pre_election: true
I20260809 15:07:20.975454 28105 raft_consensus.cc:2454] T de89bfb4cdeb49639de6b2d94bb1c70b P 98aa81f8143041c2b4d3ca9152d8786d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f13918c70fa446eb99ea081efc105f8d in term 0.
I20260809 15:07:20.975455 28529 raft_consensus.cc:2454] T de89bfb4cdeb49639de6b2d94bb1c70b P 0b6648d303b44cf39b7e8dad629f4c3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f13918c70fa446eb99ea081efc105f8d in term 0.
I20260809 15:07:20.975950 28200 leader_election.cc:302] T de89bfb4cdeb49639de6b2d94bb1c70b P f13918c70fa446eb99ea081efc105f8d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b6648d303b44cf39b7e8dad629f4c3a, f13918c70fa446eb99ea081efc105f8d; no voters:
I20260809 15:07:20.976243 28778 raft_consensus.cc:2781] T de89bfb4cdeb49639de6b2d94bb1c70b P f13918c70fa446eb99ea081efc105f8d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:07:20.976380 28778 raft_consensus.cc:482] T de89bfb4cdeb49639de6b2d94bb1c70b P f13918c70fa446eb99ea081efc105f8d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:07:20.976492 28778 raft_consensus.cc:3037] T de89bfb4cdeb49639de6b2d94bb1c70b P f13918c70fa446eb99ea081efc105f8d [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:20.978624 28778 raft_consensus.cc:504] T de89bfb4cdeb49639de6b2d94bb1c70b P f13918c70fa446eb99ea081efc105f8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } }
I20260809 15:07:20.979131 28778 leader_election.cc:288] T de89bfb4cdeb49639de6b2d94bb1c70b P f13918c70fa446eb99ea081efc105f8d [CANDIDATE]: Term 1 election: Requested vote from peers 98aa81f8143041c2b4d3ca9152d8786d (127.24.183.193:39591), 0b6648d303b44cf39b7e8dad629f4c3a (127.24.183.195:35827)
I20260809 15:07:20.979631 28105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de89bfb4cdeb49639de6b2d94bb1c70b" candidate_uuid: "f13918c70fa446eb99ea081efc105f8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98aa81f8143041c2b4d3ca9152d8786d"
I20260809 15:07:20.979627 28529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de89bfb4cdeb49639de6b2d94bb1c70b" candidate_uuid: "f13918c70fa446eb99ea081efc105f8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b6648d303b44cf39b7e8dad629f4c3a"
I20260809 15:07:20.979821 28105 raft_consensus.cc:3037] T de89bfb4cdeb49639de6b2d94bb1c70b P 98aa81f8143041c2b4d3ca9152d8786d [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:20.979846 28529 raft_consensus.cc:3037] T de89bfb4cdeb49639de6b2d94bb1c70b P 0b6648d303b44cf39b7e8dad629f4c3a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:20.981839 28529 raft_consensus.cc:2454] T de89bfb4cdeb49639de6b2d94bb1c70b P 0b6648d303b44cf39b7e8dad629f4c3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f13918c70fa446eb99ea081efc105f8d in term 1.
I20260809 15:07:20.981890 28105 raft_consensus.cc:2454] T de89bfb4cdeb49639de6b2d94bb1c70b P 98aa81f8143041c2b4d3ca9152d8786d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f13918c70fa446eb99ea081efc105f8d in term 1.
I20260809 15:07:20.982232 28200 leader_election.cc:302] T de89bfb4cdeb49639de6b2d94bb1c70b P f13918c70fa446eb99ea081efc105f8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b6648d303b44cf39b7e8dad629f4c3a, f13918c70fa446eb99ea081efc105f8d; no voters:
I20260809 15:07:20.982514 28778 raft_consensus.cc:2781] T de89bfb4cdeb49639de6b2d94bb1c70b P f13918c70fa446eb99ea081efc105f8d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:07:20.982961 28778 raft_consensus.cc:686] T de89bfb4cdeb49639de6b2d94bb1c70b P f13918c70fa446eb99ea081efc105f8d [term 1 LEADER]: Becoming Leader. State: Replica: f13918c70fa446eb99ea081efc105f8d, State: Running, Role: LEADER
I20260809 15:07:20.983253 28778 consensus_queue.cc:237] T de89bfb4cdeb49639de6b2d94bb1c70b P f13918c70fa446eb99ea081efc105f8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } }
I20260809 15:07:20.985778 27863 catalog_manager.cc:5495] T de89bfb4cdeb49639de6b2d94bb1c70b P f13918c70fa446eb99ea081efc105f8d reported cstate change: term changed from 0 to 1, leader changed from <none> to f13918c70fa446eb99ea081efc105f8d (127.24.183.194). New cstate: current_term: 1 leader_uuid: "f13918c70fa446eb99ea081efc105f8d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } health_report { overall_health: UNKNOWN } } }
I20260809 15:07:21.030849 28783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:21.025802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37690 (local address 127.24.183.194:46275)
0809 15:07:21.026380 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:21.026385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:21.026432 (+ 47us) server_negotiation.cc:407] Connection header received
0809 15:07:21.026493 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:21.026498 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:21.026562 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:21.026649 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:07:21.027048 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:21.027797 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:21.029086 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:21.029544 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:21.029653 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:21.030077 (+ 424us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:07:21.030189 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:07:21.030265 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:21.030442 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 15:07:21.030521 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":317,"threads_started":1}
I20260809 15:07:21.038787 28105 raft_consensus.cc:1261] T de89bfb4cdeb49639de6b2d94bb1c70b P 98aa81f8143041c2b4d3ca9152d8786d [term 1 FOLLOWER]: Refusing update from remote peer f13918c70fa446eb99ea081efc105f8d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:07:21.038995 28529 raft_consensus.cc:1261] T de89bfb4cdeb49639de6b2d94bb1c70b P 0b6648d303b44cf39b7e8dad629f4c3a [term 1 FOLLOWER]: Refusing update from remote peer f13918c70fa446eb99ea081efc105f8d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:07:21.039427 28779 consensus_queue.cc:1035] T de89bfb4cdeb49639de6b2d94bb1c70b P f13918c70fa446eb99ea081efc105f8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:07:21.039753 28778 consensus_queue.cc:1035] T de89bfb4cdeb49639de6b2d94bb1c70b P f13918c70fa446eb99ea081efc105f8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:07:21.043469 28721 mvcc.cc:205] Tried to move back new op lower bound from 7316635816089800704 to 7316635815869788160. Current Snapshot: MvccSnapshot[applied={T|T < 7316635816089800704}]
I20260809 15:07:21.083909 27860 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58708:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:07:21.084127 27860 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58708:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:07:21.086992 27860 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9b8c4d0b7e114003ab0f7e0a059f798f: Sending DeleteTablet for 3 replicas of tablet de89bfb4cdeb49639de6b2d94bb1c70b
I20260809 15:07:21.087627 28068 tablet_service.cc:1514] Processing DeleteTablet for tablet de89bfb4cdeb49639de6b2d94bb1c70b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:07:21 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:55356
I20260809 15:07:21.087754 28278 tablet_service.cc:1514] Processing DeleteTablet for tablet de89bfb4cdeb49639de6b2d94bb1c70b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:07:21 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:37686
I20260809 15:07:21.087895 28476 tablet_service.cc:1514] Processing DeleteTablet for tablet de89bfb4cdeb49639de6b2d94bb1c70b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:07:21 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:53072
I20260809 15:07:21.088035 28754 tablet_replica.cc:330] T de89bfb4cdeb49639de6b2d94bb1c70b P f13918c70fa446eb99ea081efc105f8d: stopping tablet replica
I20260809 15:07:21.088172 28747 tablet_replica.cc:330] T de89bfb4cdeb49639de6b2d94bb1c70b P 98aa81f8143041c2b4d3ca9152d8786d: stopping tablet replica
I20260809 15:07:21.088238 28754 raft_consensus.cc:2229] T de89bfb4cdeb49639de6b2d94bb1c70b P f13918c70fa446eb99ea081efc105f8d [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:07:21.088371 28747 raft_consensus.cc:2229] T de89bfb4cdeb49639de6b2d94bb1c70b P 98aa81f8143041c2b4d3ca9152d8786d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:07:21.088555 28752 tablet_replica.cc:330] T de89bfb4cdeb49639de6b2d94bb1c70b P 0b6648d303b44cf39b7e8dad629f4c3a: stopping tablet replica
I20260809 15:07:21.088563 28747 raft_consensus.cc:2258] T de89bfb4cdeb49639de6b2d94bb1c70b P 98aa81f8143041c2b4d3ca9152d8786d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:07:21.088572 28754 raft_consensus.cc:2258] T de89bfb4cdeb49639de6b2d94bb1c70b P f13918c70fa446eb99ea081efc105f8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:07:21.088748 28752 raft_consensus.cc:2229] T de89bfb4cdeb49639de6b2d94bb1c70b P 0b6648d303b44cf39b7e8dad629f4c3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:07:21.088936 28752 raft_consensus.cc:2258] T de89bfb4cdeb49639de6b2d94bb1c70b P 0b6648d303b44cf39b7e8dad629f4c3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:07:21.089051 27860 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58708:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 15:07:21.089707 28754 ts_tablet_manager.cc:1905] T de89bfb4cdeb49639de6b2d94bb1c70b P f13918c70fa446eb99ea081efc105f8d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:07:21.089722 28747 ts_tablet_manager.cc:1905] T de89bfb4cdeb49639de6b2d94bb1c70b P 98aa81f8143041c2b4d3ca9152d8786d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 15:07:21.089972 27860 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:07:21.090029 28752 ts_tablet_manager.cc:1905] T de89bfb4cdeb49639de6b2d94bb1c70b P 0b6648d303b44cf39b7e8dad629f4c3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:07:21.095073 28747 ts_tablet_manager.cc:1918] T de89bfb4cdeb49639de6b2d94bb1c70b P 98aa81f8143041c2b4d3ca9152d8786d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:07:21.095217 28747 log.cc:1196] T de89bfb4cdeb49639de6b2d94bb1c70b P 98aa81f8143041c2b4d3ca9152d8786d: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-0/wal/wals/de89bfb4cdeb49639de6b2d94bb1c70b
I20260809 15:07:21.095567 28747 ts_tablet_manager.cc:1939] T de89bfb4cdeb49639de6b2d94bb1c70b P 98aa81f8143041c2b4d3ca9152d8786d: Deleting consensus metadata
I20260809 15:07:21.096184 28752 ts_tablet_manager.cc:1918] T de89bfb4cdeb49639de6b2d94bb1c70b P 0b6648d303b44cf39b7e8dad629f4c3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:07:21.096189 28754 ts_tablet_manager.cc:1918] T de89bfb4cdeb49639de6b2d94bb1c70b P f13918c70fa446eb99ea081efc105f8d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:07:21.096342 28754 log.cc:1196] T de89bfb4cdeb49639de6b2d94bb1c70b P f13918c70fa446eb99ea081efc105f8d: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-1/wal/wals/de89bfb4cdeb49639de6b2d94bb1c70b
I20260809 15:07:21.096360 28752 log.cc:1196] T de89bfb4cdeb49639de6b2d94bb1c70b P 0b6648d303b44cf39b7e8dad629f4c3a: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-2/wal/wals/de89bfb4cdeb49639de6b2d94bb1c70b
I20260809 15:07:21.096747 28752 ts_tablet_manager.cc:1939] T de89bfb4cdeb49639de6b2d94bb1c70b P 0b6648d303b44cf39b7e8dad629f4c3a: Deleting consensus metadata
I20260809 15:07:21.096747 28754 ts_tablet_manager.cc:1939] T de89bfb4cdeb49639de6b2d94bb1c70b P f13918c70fa446eb99ea081efc105f8d: Deleting consensus metadata
I20260809 15:07:21.097409 27840 catalog_manager.cc:4841] TS 98aa81f8143041c2b4d3ca9152d8786d (127.24.183.193:39591): tablet de89bfb4cdeb49639de6b2d94bb1c70b (table test-table [id=626b5b89ef494af4ac99b7ba089268d4]) successfully deleted
I20260809 15:07:21.097743 27840 catalog_manager.cc:4841] TS f13918c70fa446eb99ea081efc105f8d (127.24.183.194:46275): tablet de89bfb4cdeb49639de6b2d94bb1c70b (table test-table [id=626b5b89ef494af4ac99b7ba089268d4]) successfully deleted
I20260809 15:07:21.097829 27838 catalog_manager.cc:4841] TS 0b6648d303b44cf39b7e8dad629f4c3a (127.24.183.195:35827): tablet de89bfb4cdeb49639de6b2d94bb1c70b (table test-table [id=626b5b89ef494af4ac99b7ba089268d4]) successfully deleted
I20260809 15:07:21.100800 28068 tablet_service.cc:1467] Processing CreateTablet for tablet 20ed699c166d4a8585ad3b6433d5c842 (DEFAULT_TABLE table=test-table [id=e3ae2634579b4dd1b7163e1b7924ee5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:07:21.100875 28278 tablet_service.cc:1467] Processing CreateTablet for tablet 20ed699c166d4a8585ad3b6433d5c842 (DEFAULT_TABLE table=test-table [id=e3ae2634579b4dd1b7163e1b7924ee5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:07:21.101001 28476 tablet_service.cc:1467] Processing CreateTablet for tablet 20ed699c166d4a8585ad3b6433d5c842 (DEFAULT_TABLE table=test-table [id=e3ae2634579b4dd1b7163e1b7924ee5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:07:21.101210 28068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20ed699c166d4a8585ad3b6433d5c842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:21.101210 28278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20ed699c166d4a8585ad3b6433d5c842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:21.101380 28476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20ed699c166d4a8585ad3b6433d5c842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:21.105401 28768 tablet_bootstrap.cc:492] T 20ed699c166d4a8585ad3b6433d5c842 P 0b6648d303b44cf39b7e8dad629f4c3a: Bootstrap starting.
I20260809 15:07:21.106704 28768 tablet_bootstrap.cc:654] T 20ed699c166d4a8585ad3b6433d5c842 P 0b6648d303b44cf39b7e8dad629f4c3a: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:21.107265 28766 tablet_bootstrap.cc:492] T 20ed699c166d4a8585ad3b6433d5c842 P f13918c70fa446eb99ea081efc105f8d: Bootstrap starting.
I20260809 15:07:21.107869 28767 tablet_bootstrap.cc:492] T 20ed699c166d4a8585ad3b6433d5c842 P 98aa81f8143041c2b4d3ca9152d8786d: Bootstrap starting.
I20260809 15:07:21.108798 28768 tablet_bootstrap.cc:492] T 20ed699c166d4a8585ad3b6433d5c842 P 0b6648d303b44cf39b7e8dad629f4c3a: No bootstrap required, opened a new log
I20260809 15:07:21.108966 28768 ts_tablet_manager.cc:1397] T 20ed699c166d4a8585ad3b6433d5c842 P 0b6648d303b44cf39b7e8dad629f4c3a: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260809 15:07:21.109479 28767 tablet_bootstrap.cc:654] T 20ed699c166d4a8585ad3b6433d5c842 P 98aa81f8143041c2b4d3ca9152d8786d: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:21.109731 28768 raft_consensus.cc:348] T 20ed699c166d4a8585ad3b6433d5c842 P 0b6648d303b44cf39b7e8dad629f4c3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } }
I20260809 15:07:21.109992 28768 raft_consensus.cc:374] T 20ed699c166d4a8585ad3b6433d5c842 P 0b6648d303b44cf39b7e8dad629f4c3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:21.110077 28768 raft_consensus.cc:729] T 20ed699c166d4a8585ad3b6433d5c842 P 0b6648d303b44cf39b7e8dad629f4c3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b6648d303b44cf39b7e8dad629f4c3a, State: Initialized, Role: FOLLOWER
I20260809 15:07:21.110318 28768 consensus_queue.cc:260] T 20ed699c166d4a8585ad3b6433d5c842 P 0b6648d303b44cf39b7e8dad629f4c3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } }
I20260809 15:07:21.110878 28768 ts_tablet_manager.cc:1428] T 20ed699c166d4a8585ad3b6433d5c842 P 0b6648d303b44cf39b7e8dad629f4c3a: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 15:07:21.111410 28766 tablet_bootstrap.cc:654] T 20ed699c166d4a8585ad3b6433d5c842 P f13918c70fa446eb99ea081efc105f8d: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:21.112689 28767 tablet_bootstrap.cc:492] T 20ed699c166d4a8585ad3b6433d5c842 P 98aa81f8143041c2b4d3ca9152d8786d: No bootstrap required, opened a new log
I20260809 15:07:21.112846 28767 ts_tablet_manager.cc:1397] T 20ed699c166d4a8585ad3b6433d5c842 P 98aa81f8143041c2b4d3ca9152d8786d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 15:07:21.113555 28767 raft_consensus.cc:348] T 20ed699c166d4a8585ad3b6433d5c842 P 98aa81f8143041c2b4d3ca9152d8786d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } }
I20260809 15:07:21.113807 28767 raft_consensus.cc:374] T 20ed699c166d4a8585ad3b6433d5c842 P 98aa81f8143041c2b4d3ca9152d8786d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:21.113895 28767 raft_consensus.cc:729] T 20ed699c166d4a8585ad3b6433d5c842 P 98aa81f8143041c2b4d3ca9152d8786d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98aa81f8143041c2b4d3ca9152d8786d, State: Initialized, Role: FOLLOWER
I20260809 15:07:21.114166 28767 consensus_queue.cc:260] T 20ed699c166d4a8585ad3b6433d5c842 P 98aa81f8143041c2b4d3ca9152d8786d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } }
I20260809 15:07:21.114682 28767 ts_tablet_manager.cc:1428] T 20ed699c166d4a8585ad3b6433d5c842 P 98aa81f8143041c2b4d3ca9152d8786d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 15:07:21.116364 28766 tablet_bootstrap.cc:492] T 20ed699c166d4a8585ad3b6433d5c842 P f13918c70fa446eb99ea081efc105f8d: No bootstrap required, opened a new log
I20260809 15:07:21.116526 28766 ts_tablet_manager.cc:1397] T 20ed699c166d4a8585ad3b6433d5c842 P f13918c70fa446eb99ea081efc105f8d: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.004s
I20260809 15:07:21.117120 28766 raft_consensus.cc:348] T 20ed699c166d4a8585ad3b6433d5c842 P f13918c70fa446eb99ea081efc105f8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } }
I20260809 15:07:21.117306 28766 raft_consensus.cc:374] T 20ed699c166d4a8585ad3b6433d5c842 P f13918c70fa446eb99ea081efc105f8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:21.117385 28766 raft_consensus.cc:729] T 20ed699c166d4a8585ad3b6433d5c842 P f13918c70fa446eb99ea081efc105f8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f13918c70fa446eb99ea081efc105f8d, State: Initialized, Role: FOLLOWER
I20260809 15:07:21.117611 28766 consensus_queue.cc:260] T 20ed699c166d4a8585ad3b6433d5c842 P f13918c70fa446eb99ea081efc105f8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } }
I20260809 15:07:21.118180 28766 ts_tablet_manager.cc:1428] T 20ed699c166d4a8585ad3b6433d5c842 P f13918c70fa446eb99ea081efc105f8d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 15:07:21.276499 28791 raft_consensus.cc:482] T 20ed699c166d4a8585ad3b6433d5c842 P f13918c70fa446eb99ea081efc105f8d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:07:21.276702 28791 raft_consensus.cc:504] T 20ed699c166d4a8585ad3b6433d5c842 P f13918c70fa446eb99ea081efc105f8d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } }
I20260809 15:07:21.277513 28791 leader_election.cc:288] T 20ed699c166d4a8585ad3b6433d5c842 P f13918c70fa446eb99ea081efc105f8d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98aa81f8143041c2b4d3ca9152d8786d (127.24.183.193:39591), 0b6648d303b44cf39b7e8dad629f4c3a (127.24.183.195:35827)
I20260809 15:07:21.278167 28105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20ed699c166d4a8585ad3b6433d5c842" candidate_uuid: "f13918c70fa446eb99ea081efc105f8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98aa81f8143041c2b4d3ca9152d8786d" is_pre_election: true
I20260809 15:07:21.278167 28526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20ed699c166d4a8585ad3b6433d5c842" candidate_uuid: "f13918c70fa446eb99ea081efc105f8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" is_pre_election: true
I20260809 15:07:21.278442 28526 raft_consensus.cc:2454] T 20ed699c166d4a8585ad3b6433d5c842 P 0b6648d303b44cf39b7e8dad629f4c3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f13918c70fa446eb99ea081efc105f8d in term 0.
I20260809 15:07:21.278442 28105 raft_consensus.cc:2454] T 20ed699c166d4a8585ad3b6433d5c842 P 98aa81f8143041c2b4d3ca9152d8786d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f13918c70fa446eb99ea081efc105f8d in term 0.
I20260809 15:07:21.278885 28205 leader_election.cc:302] T 20ed699c166d4a8585ad3b6433d5c842 P f13918c70fa446eb99ea081efc105f8d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98aa81f8143041c2b4d3ca9152d8786d, f13918c70fa446eb99ea081efc105f8d; no voters:
I20260809 15:07:21.279183 28791 raft_consensus.cc:2781] T 20ed699c166d4a8585ad3b6433d5c842 P f13918c70fa446eb99ea081efc105f8d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:07:21.279304 28791 raft_consensus.cc:482] T 20ed699c166d4a8585ad3b6433d5c842 P f13918c70fa446eb99ea081efc105f8d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:07:21.279394 28791 raft_consensus.cc:3037] T 20ed699c166d4a8585ad3b6433d5c842 P f13918c70fa446eb99ea081efc105f8d [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:21.281591 28791 raft_consensus.cc:504] T 20ed699c166d4a8585ad3b6433d5c842 P f13918c70fa446eb99ea081efc105f8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } }
I20260809 15:07:21.282128 28791 leader_election.cc:288] T 20ed699c166d4a8585ad3b6433d5c842 P f13918c70fa446eb99ea081efc105f8d [CANDIDATE]: Term 1 election: Requested vote from peers 98aa81f8143041c2b4d3ca9152d8786d (127.24.183.193:39591), 0b6648d303b44cf39b7e8dad629f4c3a (127.24.183.195:35827)
I20260809 15:07:21.282670 28526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20ed699c166d4a8585ad3b6433d5c842" candidate_uuid: "f13918c70fa446eb99ea081efc105f8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b6648d303b44cf39b7e8dad629f4c3a"
I20260809 15:07:21.282670 28105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20ed699c166d4a8585ad3b6433d5c842" candidate_uuid: "f13918c70fa446eb99ea081efc105f8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98aa81f8143041c2b4d3ca9152d8786d"
I20260809 15:07:21.282893 28526 raft_consensus.cc:3037] T 20ed699c166d4a8585ad3b6433d5c842 P 0b6648d303b44cf39b7e8dad629f4c3a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:21.282912 28105 raft_consensus.cc:3037] T 20ed699c166d4a8585ad3b6433d5c842 P 98aa81f8143041c2b4d3ca9152d8786d [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:21.285897 28105 raft_consensus.cc:2454] T 20ed699c166d4a8585ad3b6433d5c842 P 98aa81f8143041c2b4d3ca9152d8786d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f13918c70fa446eb99ea081efc105f8d in term 1.
I20260809 15:07:21.285888 28526 raft_consensus.cc:2454] T 20ed699c166d4a8585ad3b6433d5c842 P 0b6648d303b44cf39b7e8dad629f4c3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f13918c70fa446eb99ea081efc105f8d in term 1.
I20260809 15:07:21.286350 28205 leader_election.cc:302] T 20ed699c166d4a8585ad3b6433d5c842 P f13918c70fa446eb99ea081efc105f8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0b6648d303b44cf39b7e8dad629f4c3a, 98aa81f8143041c2b4d3ca9152d8786d, f13918c70fa446eb99ea081efc105f8d; no voters:
I20260809 15:07:21.286649 28791 raft_consensus.cc:2781] T 20ed699c166d4a8585ad3b6433d5c842 P f13918c70fa446eb99ea081efc105f8d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:07:21.286844 28791 raft_consensus.cc:686] T 20ed699c166d4a8585ad3b6433d5c842 P f13918c70fa446eb99ea081efc105f8d [term 1 LEADER]: Becoming Leader. State: Replica: f13918c70fa446eb99ea081efc105f8d, State: Running, Role: LEADER
I20260809 15:07:21.287140 28791 consensus_queue.cc:237] T 20ed699c166d4a8585ad3b6433d5c842 P f13918c70fa446eb99ea081efc105f8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } }
I20260809 15:07:21.289817 27860 catalog_manager.cc:5495] T 20ed699c166d4a8585ad3b6433d5c842 P f13918c70fa446eb99ea081efc105f8d reported cstate change: term changed from 0 to 1, leader changed from <none> to f13918c70fa446eb99ea081efc105f8d (127.24.183.194). New cstate: current_term: 1 leader_uuid: "f13918c70fa446eb99ea081efc105f8d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f13918c70fa446eb99ea081efc105f8d" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 } health_report { overall_health: UNKNOWN } } }
I20260809 15:07:21.316658 28105 raft_consensus.cc:1261] T 20ed699c166d4a8585ad3b6433d5c842 P 98aa81f8143041c2b4d3ca9152d8786d [term 1 FOLLOWER]: Refusing update from remote peer f13918c70fa446eb99ea081efc105f8d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:07:21.316754 28526 raft_consensus.cc:1261] T 20ed699c166d4a8585ad3b6433d5c842 P 0b6648d303b44cf39b7e8dad629f4c3a [term 1 FOLLOWER]: Refusing update from remote peer f13918c70fa446eb99ea081efc105f8d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:07:21.317189 28791 consensus_queue.cc:1035] T 20ed699c166d4a8585ad3b6433d5c842 P f13918c70fa446eb99ea081efc105f8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "98aa81f8143041c2b4d3ca9152d8786d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:07:21.317487 28792 consensus_queue.cc:1035] T 20ed699c166d4a8585ad3b6433d5c842 P f13918c70fa446eb99ea081efc105f8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b6648d303b44cf39b7e8dad629f4c3a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 35827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 15:07:21.337779 28721 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 15:07:22.916913 27860 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58708:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:07:22.917130 27860 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58708:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:07:22.921068 27860 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9b8c4d0b7e114003ab0f7e0a059f798f: Sending DeleteTablet for 3 replicas of tablet 20ed699c166d4a8585ad3b6433d5c842
I20260809 15:07:22.921727 28068 tablet_service.cc:1514] Processing DeleteTablet for tablet 20ed699c166d4a8585ad3b6433d5c842 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:07:22 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:55356
I20260809 15:07:22.921919 28278 tablet_service.cc:1514] Processing DeleteTablet for tablet 20ed699c166d4a8585ad3b6433d5c842 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:07:22 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:37686
I20260809 15:07:22.921941 28476 tablet_service.cc:1514] Processing DeleteTablet for tablet 20ed699c166d4a8585ad3b6433d5c842 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:07:22 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:53072
I20260809 15:07:22.922556 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 27970
I20260809 15:07:22.934124 28833 tablet_replica.cc:330] T 20ed699c166d4a8585ad3b6433d5c842 P 98aa81f8143041c2b4d3ca9152d8786d: stopping tablet replica
I20260809 15:07:22.934813 28834 tablet_replica.cc:330] T 20ed699c166d4a8585ad3b6433d5c842 P f13918c70fa446eb99ea081efc105f8d: stopping tablet replica
I20260809 15:07:22.934808 28833 raft_consensus.cc:2229] T 20ed699c166d4a8585ad3b6433d5c842 P 98aa81f8143041c2b4d3ca9152d8786d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:07:22.935079 28833 raft_consensus.cc:2258] T 20ed699c166d4a8585ad3b6433d5c842 P 98aa81f8143041c2b4d3ca9152d8786d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:07:22.935293 28834 raft_consensus.cc:2229] T 20ed699c166d4a8585ad3b6433d5c842 P f13918c70fa446eb99ea081efc105f8d [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:07:22.935693 28834 raft_consensus.cc:2258] T 20ed699c166d4a8585ad3b6433d5c842 P f13918c70fa446eb99ea081efc105f8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:07:22.936335 28833 ts_tablet_manager.cc:1905] T 20ed699c166d4a8585ad3b6433d5c842 P 98aa81f8143041c2b4d3ca9152d8786d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:07:22.936820 28834 ts_tablet_manager.cc:1905] T 20ed699c166d4a8585ad3b6433d5c842 P f13918c70fa446eb99ea081efc105f8d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:07:22.943400 28834 ts_tablet_manager.cc:1918] T 20ed699c166d4a8585ad3b6433d5c842 P f13918c70fa446eb99ea081efc105f8d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 15:07:22.943400 28833 ts_tablet_manager.cc:1918] T 20ed699c166d4a8585ad3b6433d5c842 P 98aa81f8143041c2b4d3ca9152d8786d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 15:07:22.943536 28833 log.cc:1196] T 20ed699c166d4a8585ad3b6433d5c842 P 98aa81f8143041c2b4d3ca9152d8786d: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-0/wal/wals/20ed699c166d4a8585ad3b6433d5c842
I20260809 15:07:22.944008 28835 tablet_replica.cc:330] T 20ed699c166d4a8585ad3b6433d5c842 P 0b6648d303b44cf39b7e8dad629f4c3a: stopping tablet replica
I20260809 15:07:22.944156 28833 ts_tablet_manager.cc:1939] T 20ed699c166d4a8585ad3b6433d5c842 P 98aa81f8143041c2b4d3ca9152d8786d: Deleting consensus metadata
I20260809 15:07:22.944247 28834 log.cc:1196] T 20ed699c166d4a8585ad3b6433d5c842 P f13918c70fa446eb99ea081efc105f8d: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-1/wal/wals/20ed699c166d4a8585ad3b6433d5c842
I20260809 15:07:22.944235 28835 raft_consensus.cc:2229] T 20ed699c166d4a8585ad3b6433d5c842 P 0b6648d303b44cf39b7e8dad629f4c3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:07:22.944517 28835 raft_consensus.cc:2258] T 20ed699c166d4a8585ad3b6433d5c842 P 0b6648d303b44cf39b7e8dad629f4c3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:07:22.944757 28834 ts_tablet_manager.cc:1939] T 20ed699c166d4a8585ad3b6433d5c842 P f13918c70fa446eb99ea081efc105f8d: Deleting consensus metadata
I20260809 15:07:22.945467 27840 catalog_manager.cc:4841] TS 98aa81f8143041c2b4d3ca9152d8786d (127.24.183.193:39591): tablet 20ed699c166d4a8585ad3b6433d5c842 (table test-table [id=e3ae2634579b4dd1b7163e1b7924ee5f]) successfully deleted
I20260809 15:07:22.945942 27840 catalog_manager.cc:4841] TS f13918c70fa446eb99ea081efc105f8d (127.24.183.194:46275): tablet 20ed699c166d4a8585ad3b6433d5c842 (table test-table [id=e3ae2634579b4dd1b7163e1b7924ee5f]) successfully deleted
I20260809 15:07:22.945978 28835 ts_tablet_manager.cc:1905] T 20ed699c166d4a8585ad3b6433d5c842 P 0b6648d303b44cf39b7e8dad629f4c3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:07:22.950743 28835 ts_tablet_manager.cc:1918] T 20ed699c166d4a8585ad3b6433d5c842 P 0b6648d303b44cf39b7e8dad629f4c3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 15:07:22.950850 28835 log.cc:1196] T 20ed699c166d4a8585ad3b6433d5c842 P 0b6648d303b44cf39b7e8dad629f4c3a: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288021313853-25311-0/minicluster-data/ts-2/wal/wals/20ed699c166d4a8585ad3b6433d5c842
I20260809 15:07:22.951217 28835 ts_tablet_manager.cc:1939] T 20ed699c166d4a8585ad3b6433d5c842 P 0b6648d303b44cf39b7e8dad629f4c3a: Deleting consensus metadata
Aug 09 15:07:22 dist-test-slave-nj21 krb5kdc[27791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288040, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.183.193@KRBTEST.COM
I20260809 15:07:22.952294 27838 catalog_manager.cc:4841] TS 0b6648d303b44cf39b7e8dad629f4c3a (127.24.183.195:35827): tablet 20ed699c166d4a8585ad3b6433d5c842 (table test-table [id=e3ae2634579b4dd1b7163e1b7924ee5f]) successfully deleted
I20260809 15:07:22.958551 28836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:22.923916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55396 (local address 127.24.183.193:39591)
0809 15:07:22.939283 (+ 15367us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:22.939290 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:22.939318 (+ 28us) server_negotiation.cc:407] Connection header received
0809 15:07:22.939386 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:22.939393 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:22.939478 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:22.941612 (+ 2134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:22.941873 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:22.943123 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:22.944926 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:22.945690 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:22.945829 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:22.953314 (+ 7485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:22.953332 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:22.953352 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:22.953386 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:22.956087 (+ 2701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:22.956727 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:22.956734 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:22.956737 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:22.956912 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:22.957262 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:22.957265 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:22.957267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:22.957650 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:22.957912 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:22.958092 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 15:07:22.958280 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15220,"thread_start_us":15679,"threads_started":1}
W20260809 15:07:22.959374 28162 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55396
W20260809 15:07:22.960078 25311 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 27970: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:07:22.960289 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 27970
I20260809 15:07:22.996563 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 28176
Aug 09 15:07:23 dist-test-slave-nj21 krb5kdc[27791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288040, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.183.194@KRBTEST.COM
I20260809 15:07:23.010704 28839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:22.997290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37698 (local address 127.24.183.194:46275)
0809 15:07:22.997804 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:22.997809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:22.997826 (+ 17us) server_negotiation.cc:407] Connection header received
0809 15:07:22.997892 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:22.997896 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:22.997965 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:22.998063 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:22.999203 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:23.000014 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:23.001176 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:23.001562 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:23.001649 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:23.005616 (+ 3967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:23.005631 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:23.005646 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:23.005679 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:23.008536 (+ 2857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:23.009066 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:23.009072 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:23.009074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:23.009191 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:23.009453 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:23.009456 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:23.009458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:23.009805 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:23.010038 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:23.010201 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 15:07:23.010371 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":282,"threads_started":1}
W20260809 15:07:23.011346 28368 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37698
W20260809 15:07:23.011958 25311 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 28176: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:07:23.012094 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 28176
I20260809 15:07:23.040185 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 28382
Aug 09 15:07:23 dist-test-slave-nj21 krb5kdc[27791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288040, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.183.195@KRBTEST.COM
I20260809 15:07:23.055059 28840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:23.040924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53080 (local address 127.24.183.195:35827)
0809 15:07:23.041481 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:23.041486 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:23.041502 (+ 16us) server_negotiation.cc:407] Connection header received
0809 15:07:23.041559 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:23.041563 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:23.041632 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:23.041724 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:23.043121 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:23.043951 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:23.045182 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:23.045559 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:23.045645 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:23.049768 (+ 4123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:23.049783 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:23.049801 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:23.049845 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:23.052849 (+ 3004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:23.053422 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:23.053429 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:23.053431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:23.053570 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:23.053848 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:23.053851 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:23.053853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:23.054180 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:23.054398 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:23.054591 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 15:07:23.054758 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":299,"threads_started":1}
W20260809 15:07:23.055886 28590 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53080
W20260809 15:07:23.056499 25311 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 28382: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:07:23.056654 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 28382
I20260809 15:07:23.081233 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 27807
I20260809 15:07:23.082448 27924 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:07:23.202394 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 27807
2026-08-09T15:07:23Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8765 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:07:23 dist-test-slave-nj21 krb5kdc[28846](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:07:23 dist-test-slave-nj21 krb5kdc[28846](info): set up 2 sockets
Aug 09 15:07:23 dist-test-slave-nj21 krb5kdc[28846](info): commencing operation
krb5kdc: starting...
W20260809 15:07:24.423888 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.005s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:07:24 dist-test-slave-nj21 krb5kdc[28846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:07:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:07:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
I20260809 15:07:24.613229 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:38131
--webserver_interface=127.24.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:37605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:38131
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:24.935508 28868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:24.935788 28868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:24.935923 28868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:24.943071 28868 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:07:24.943178 28868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:24.943251 28868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:24.943331 28868 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:07:24.943389 28868 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:07:24.953493 28868 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:37605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:38131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:38131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:24.954927 28868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:24.956291 28868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:24.968259 28876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:24.968216 28880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:24.968315 28877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:24.968726 28868 server_base.cc:1034] running on GCE node
I20260809 15:07:24.969753 28868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:24.971153 28868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:24.972399 28868 hybrid_clock.cc:648] HybridClock initialized: now 1786288044972343 us; error 33 us; skew 500 ppm
I20260809 15:07:24.972800 28868 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:24 dist-test-slave-nj21 krb5kdc[28846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288044, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:24.975982 28868 init.cc:377] Logged in from keytab as kudu/127.24.183.254@KRBTEST.COM (short username kudu)
I20260809 15:07:24.977458 28868 webserver.cc:460] Webserver started at http://127.24.183.254:45589/ using document root <none> and password file <none>
I20260809 15:07:24.978137 28868 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:24.978255 28868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:24.978565 28868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:24.981619 28868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "01c989d3952942b9b8ebc33ba60a0a40"
format_stamp: "Formatted at 2026-08-09 15:07:24 on dist-test-slave-nj21"
server_key: "51f43c3e8f86f8495cd9375a5c7c61a7"
server_key_iv: "bd684d1c21ab3bf25c631cd3f0c8d906"
server_key_version: "encryptionkey@0"
I20260809 15:07:24.982372 28868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "01c989d3952942b9b8ebc33ba60a0a40"
format_stamp: "Formatted at 2026-08-09 15:07:24 on dist-test-slave-nj21"
server_key: "51f43c3e8f86f8495cd9375a5c7c61a7"
server_key_iv: "bd684d1c21ab3bf25c631cd3f0c8d906"
server_key_version: "encryptionkey@0"
I20260809 15:07:24.987479 28868 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 15:07:24.990799 28893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:24.992255 28868 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:07:24.992424 28868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "01c989d3952942b9b8ebc33ba60a0a40"
format_stamp: "Formatted at 2026-08-09 15:07:24 on dist-test-slave-nj21"
server_key: "51f43c3e8f86f8495cd9375a5c7c61a7"
server_key_iv: "bd684d1c21ab3bf25c631cd3f0c8d906"
server_key_version: "encryptionkey@0"
I20260809 15:07:24.992607 28868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:25.018023 28868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:25.022214 28868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:25.022501 28868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:25.040318 28868 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:38131
I20260809 15:07:25.040338 28977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:38131 every 8 connection(s)
I20260809 15:07:25.041720 28868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:07:25.045526 28978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:25.050372 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 28868
I20260809 15:07:25.050583 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:07:25.051203 25311 external_mini_cluster.cc:1428] Setting key 7bde1614a5acd26376f31d7076564b8d
I20260809 15:07:25.056425 28978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01c989d3952942b9b8ebc33ba60a0a40: Bootstrap starting.
I20260809 15:07:25.060144 28978 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01c989d3952942b9b8ebc33ba60a0a40: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:25.061368 28978 log.cc:824] T 00000000000000000000000000000000 P 01c989d3952942b9b8ebc33ba60a0a40: Log is configured to *not* fsync() on all Append() calls
I20260809 15:07:25.064764 28978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01c989d3952942b9b8ebc33ba60a0a40: No bootstrap required, opened a new log
Aug 09 15:07:25 dist-test-slave-nj21 krb5kdc[28846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:25.073571 28978 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01c989d3952942b9b8ebc33ba60a0a40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01c989d3952942b9b8ebc33ba60a0a40" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 38131 } }
I20260809 15:07:25.074035 28978 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01c989d3952942b9b8ebc33ba60a0a40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:25.074141 28978 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01c989d3952942b9b8ebc33ba60a0a40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01c989d3952942b9b8ebc33ba60a0a40, State: Initialized, Role: FOLLOWER
I20260809 15:07:25.074719 28978 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01c989d3952942b9b8ebc33ba60a0a40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01c989d3952942b9b8ebc33ba60a0a40" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 38131 } }
I20260809 15:07:25.074971 28978 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01c989d3952942b9b8ebc33ba60a0a40 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:07:25.074980 28982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:25.053139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35752 (local address 127.24.183.254:38131)
0809 15:07:25.054017 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:25.054029 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:25.054073 (+ 44us) server_negotiation.cc:407] Connection header received
0809 15:07:25.055819 (+ 1746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:25.055845 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:25.056203 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:25.056525 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:25.058345 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:25.059667 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:25.061429 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:25.062410 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:25.062602 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:25.067169 (+ 4567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:25.067180 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:25.067194 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:25.067232 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:25.070298 (+ 3066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:25.071111 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:25.071118 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:25.071123 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:25.071286 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:25.071727 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:25.071731 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:25.071734 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:25.072478 (+ 744us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:25.072758 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:25.073435 (+ 677us) server_negotiation.cc:299] Negotiation successful
0809 15:07:25.073854 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":399,"threads_started":1}
I20260809 15:07:25.075121 28978 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01c989d3952942b9b8ebc33ba60a0a40 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:07:25.075340 28978 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01c989d3952942b9b8ebc33ba60a0a40 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:25.078066 28978 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01c989d3952942b9b8ebc33ba60a0a40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01c989d3952942b9b8ebc33ba60a0a40" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 38131 } }
I20260809 15:07:25.078509 28978 leader_election.cc:302] T 00000000000000000000000000000000 P 01c989d3952942b9b8ebc33ba60a0a40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01c989d3952942b9b8ebc33ba60a0a40; no voters:
I20260809 15:07:25.079042 28978 leader_election.cc:288] T 00000000000000000000000000000000 P 01c989d3952942b9b8ebc33ba60a0a40 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:07:25.079154 28984 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01c989d3952942b9b8ebc33ba60a0a40 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:07:25.079809 28984 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01c989d3952942b9b8ebc33ba60a0a40 [term 1 LEADER]: Becoming Leader. State: Replica: 01c989d3952942b9b8ebc33ba60a0a40, State: Running, Role: LEADER
I20260809 15:07:25.080444 28984 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01c989d3952942b9b8ebc33ba60a0a40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01c989d3952942b9b8ebc33ba60a0a40" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 38131 } }
I20260809 15:07:25.081063 28978 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01c989d3952942b9b8ebc33ba60a0a40 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:07:25.082979 28986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01c989d3952942b9b8ebc33ba60a0a40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01c989d3952942b9b8ebc33ba60a0a40. Latest consensus state: current_term: 1 leader_uuid: "01c989d3952942b9b8ebc33ba60a0a40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01c989d3952942b9b8ebc33ba60a0a40" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 38131 } } }
I20260809 15:07:25.083051 28985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01c989d3952942b9b8ebc33ba60a0a40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01c989d3952942b9b8ebc33ba60a0a40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01c989d3952942b9b8ebc33ba60a0a40" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 38131 } } }
I20260809 15:07:25.083381 28986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01c989d3952942b9b8ebc33ba60a0a40 [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:25.083406 28985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01c989d3952942b9b8ebc33ba60a0a40 [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:25.086499 28991 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:07:25.091415 28991 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:07:25.100697 28991 catalog_manager.cc:1347] Generated new cluster ID: 795556320de64163bd409d1169a035c7
I20260809 15:07:25.100792 28991 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:07:25.126758 28991 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:07:25.128134 28991 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:07:25.134799 28991 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01c989d3952942b9b8ebc33ba60a0a40: Generated new TSK 0
I20260809 15:07:25.135622 28991 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab.
Entry for principal kudu/127.24.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab.
I20260809 15:07:25.199302 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.193:0
--local_ip_for_outbound_sockets=127.24.183.193
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:38131
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:37605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:25.549484 29011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:25.549773 29011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:25.549921 29011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:25.557103 29011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:25.557222 29011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:25.557380 29011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.193
I20260809 15:07:25.566928 29011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:37605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:38131
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.193
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:25.568236 29011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:25.569576 29011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:25.582176 29022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:25.582141 29025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:25.582535 29027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:25.583159 29011 server_base.cc:1034] running on GCE node
I20260809 15:07:25.583840 29011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:25.584743 29011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:25.585947 29011 hybrid_clock.cc:648] HybridClock initialized: now 1786288045585888 us; error 47 us; skew 500 ppm
I20260809 15:07:25.586246 29011 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:25 dist-test-slave-nj21 krb5kdc[28846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288045, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:25.589336 29011 init.cc:377] Logged in from keytab as kudu/127.24.183.193@KRBTEST.COM (short username kudu)
I20260809 15:07:25.590720 29011 webserver.cc:460] Webserver started at http://127.24.183.193:44995/ using document root <none> and password file <none>
I20260809 15:07:25.591387 29011 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:25.591499 29011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:25.591835 29011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:25.594758 29011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-0/data/instance:
uuid: "6d26b98fd8ba4abd872032e6b8230915"
format_stamp: "Formatted at 2026-08-09 15:07:25 on dist-test-slave-nj21"
server_key: "a6315c1ea728aff019a19b11a2d0efd2"
server_key_iv: "3e550b992ec5e94db8a9e4437b639db5"
server_key_version: "encryptionkey@0"
I20260809 15:07:25.595527 29011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance:
uuid: "6d26b98fd8ba4abd872032e6b8230915"
format_stamp: "Formatted at 2026-08-09 15:07:25 on dist-test-slave-nj21"
server_key: "a6315c1ea728aff019a19b11a2d0efd2"
server_key_iv: "3e550b992ec5e94db8a9e4437b639db5"
server_key_version: "encryptionkey@0"
I20260809 15:07:25.600636 29011 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 15:07:25.603904 29036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:25.605114 29011 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 15:07:25.605266 29011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-0/wal
uuid: "6d26b98fd8ba4abd872032e6b8230915"
format_stamp: "Formatted at 2026-08-09 15:07:25 on dist-test-slave-nj21"
server_key: "a6315c1ea728aff019a19b11a2d0efd2"
server_key_iv: "3e550b992ec5e94db8a9e4437b639db5"
server_key_version: "encryptionkey@0"
I20260809 15:07:25.605461 29011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:25.651641 29011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:25.655597 29011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:25.655900 29011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:25.656885 29011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:07:25.658236 29011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:07:25.658329 29011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:25.658459 29011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:07:25.658515 29011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:25.690675 29011 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.193:36901
I20260809 15:07:25.690750 29200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.193:36901 every 8 connection(s)
I20260809 15:07:25.692024 29011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
I20260809 15:07:25.701440 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 29011
I20260809 15:07:25.701656 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance
I20260809 15:07:25.702310 25311 external_mini_cluster.cc:1428] Setting key 8c1b76348d0285da338bb13b88fac5f8
Aug 09 15:07:25 dist-test-slave-nj21 krb5kdc[28846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288045, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:25.712229 29205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:25.694696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:36571 (local address 127.24.183.254:38131)
0809 15:07:25.695177 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:25.695182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:25.696155 (+ 973us) server_negotiation.cc:407] Connection header received
0809 15:07:25.698044 (+ 1889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:25.698051 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:25.698132 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:25.698232 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:25.699706 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:25.700499 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:25.702076 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:25.702508 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:25.702619 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:25.706478 (+ 3859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:25.706484 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:25.706487 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:25.706517 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:25.709024 (+ 2507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:25.709891 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:25.709895 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:25.709898 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:25.710019 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:25.710454 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:25.710457 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:25.710459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:25.710708 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:25.710837 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:25.711595 (+ 758us) server_negotiation.cc:299] Negotiation successful
0809 15:07:25.711871 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":235,"threads_started":1}
I20260809 15:07:25.713083 29206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:25.695272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:38131 (local address 127.24.183.193:36571)
0809 15:07:25.695996 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:25.696034 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:25.697790 (+ 1756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:25.698475 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:25.698482 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:25.698960 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:25.699521 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:25.699540 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:25.700657 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:25.700661 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:25.701941 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:25.701954 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:25.702735 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:25.702743 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:25.702970 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:25.703660 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:25.703712 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:25.706268 (+ 2556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:25.709162 (+ 2894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:25.709170 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:25.709184 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:25.709788 (+ 604us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:25.710110 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:25.710113 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:25.710116 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:25.710369 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:25.710821 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:25.710828 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:25.711238 (+ 410us) client_negotiation.cc:769] Sending connection context
0809 15:07:25.711525 (+ 287us) client_negotiation.cc:240] Negotiation successful
0809 15:07:25.711985 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":314,"threads_started":1}
I20260809 15:07:25.714532 29201 heartbeater.cc:347] Connected to a master server at 127.24.183.254:38131
I20260809 15:07:25.714840 29201 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:25.715538 29201 heartbeater.cc:511] Master 127.24.183.254:38131 requested a full tablet report, sending...
I20260809 15:07:25.717537 28925 ts_manager.cc:194] Registered new tserver with Master: 6d26b98fd8ba4abd872032e6b8230915 (127.24.183.193:36901)
I20260809 15:07:25.718979 28925 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.193@KRBTEST.COM'} at 127.24.183.193:36571
WARNING: no policy specified for kudu/127.24.183.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab.
Entry for principal kudu/127.24.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab.
I20260809 15:07:25.746649 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.194:0
--local_ip_for_outbound_sockets=127.24.183.194
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:38131
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:37605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:26.077219 29211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:26.077476 29211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:26.077584 29211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:26.084683 29211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:26.084811 29211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:26.084932 29211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.194
I20260809 15:07:26.094877 29211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:37605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:38131
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.194
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:26.096326 29211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:26.097766 29211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:26.110215 29217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:26.110316 29221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:26.110800 29219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:26.111078 29211 server_base.cc:1034] running on GCE node
I20260809 15:07:26.111706 29211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:26.112666 29211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:26.113857 29211 hybrid_clock.cc:648] HybridClock initialized: now 1786288046113791 us; error 52 us; skew 500 ppm
I20260809 15:07:26.114149 29211 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:26 dist-test-slave-nj21 krb5kdc[28846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288046, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:26.117159 29211 init.cc:377] Logged in from keytab as kudu/127.24.183.194@KRBTEST.COM (short username kudu)
I20260809 15:07:26.118458 29211 webserver.cc:460] Webserver started at http://127.24.183.194:33497/ using document root <none> and password file <none>
I20260809 15:07:26.119056 29211 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:26.119148 29211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:26.119421 29211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:26.122231 29211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-1/data/instance:
uuid: "7ea7d3b30b2e4b69a86847927081b9a2"
format_stamp: "Formatted at 2026-08-09 15:07:26 on dist-test-slave-nj21"
server_key: "031d761e6c94cf38ddefc121fd6afa0d"
server_key_iv: "f9547363af8dc181fd7a620a711dfb22"
server_key_version: "encryptionkey@0"
I20260809 15:07:26.122975 29211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance:
uuid: "7ea7d3b30b2e4b69a86847927081b9a2"
format_stamp: "Formatted at 2026-08-09 15:07:26 on dist-test-slave-nj21"
server_key: "031d761e6c94cf38ddefc121fd6afa0d"
server_key_iv: "f9547363af8dc181fd7a620a711dfb22"
server_key_version: "encryptionkey@0"
I20260809 15:07:26.127763 29211 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.001s
I20260809 15:07:26.130995 29228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:26.132275 29211 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 15:07:26.132429 29211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-1/wal
uuid: "7ea7d3b30b2e4b69a86847927081b9a2"
format_stamp: "Formatted at 2026-08-09 15:07:26 on dist-test-slave-nj21"
server_key: "031d761e6c94cf38ddefc121fd6afa0d"
server_key_iv: "f9547363af8dc181fd7a620a711dfb22"
server_key_version: "encryptionkey@0"
I20260809 15:07:26.132601 29211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:26.165688 29211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:26.169854 29211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:26.170115 29211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:26.171056 29211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:07:26.172456 29211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:07:26.172533 29211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:26.172662 29211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:07:26.172715 29211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:26.203454 29211 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.194:41583
I20260809 15:07:26.203490 29400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.194:41583 every 8 connection(s)
I20260809 15:07:26.204772 29211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
I20260809 15:07:26.214442 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 29211
I20260809 15:07:26.214622 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance
I20260809 15:07:26.215245 25311 external_mini_cluster.cc:1428] Setting key 29375c3446bee512f7c5eb0bd740d027
Aug 09 15:07:26 dist-test-slave-nj21 krb5kdc[28846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288046, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:26.223711 29205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:26.207110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:48539 (local address 127.24.183.254:38131)
0809 15:07:26.207289 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:26.207293 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:26.208433 (+ 1140us) server_negotiation.cc:407] Connection header received
0809 15:07:26.210308 (+ 1875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:26.210313 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:26.210397 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:26.210484 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:26.211908 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:26.212620 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:26.214099 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:26.214533 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:26.214652 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:26.218300 (+ 3648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:26.218305 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:26.218309 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:26.218340 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:26.220805 (+ 2465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:26.221467 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:26.221471 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:26.221473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:26.221601 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:26.222009 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:26.222013 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:26.222015 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:26.222288 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:26.222436 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:26.223105 (+ 669us) server_negotiation.cc:299] Negotiation successful
0809 15:07:26.223344 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 15:07:26.224563 29403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:26.207575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:38131 (local address 127.24.183.194:48539)
0809 15:07:26.208281 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:26.208325 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:26.210070 (+ 1745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:26.210667 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:26.210673 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:26.211168 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:26.211765 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:26.211784 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:26.212739 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:26.212742 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:26.213976 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:26.213988 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:26.214765 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:26.214773 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:26.215055 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:26.215736 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:26.215771 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:26.218090 (+ 2319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:26.220937 (+ 2847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:26.220945 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:26.220959 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:26.221344 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:26.221703 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:26.221708 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:26.221712 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:26.221897 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:26.222410 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:26.222418 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:26.222721 (+ 303us) client_negotiation.cc:769] Sending connection context
0809 15:07:26.222949 (+ 228us) client_negotiation.cc:240] Negotiation successful
0809 15:07:26.223429 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":267,"threads_started":1}
I20260809 15:07:26.225818 29401 heartbeater.cc:347] Connected to a master server at 127.24.183.254:38131
I20260809 15:07:26.226089 29401 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:26.226761 29401 heartbeater.cc:511] Master 127.24.183.254:38131 requested a full tablet report, sending...
I20260809 15:07:26.228178 28925 ts_manager.cc:194] Registered new tserver with Master: 7ea7d3b30b2e4b69a86847927081b9a2 (127.24.183.194:41583)
I20260809 15:07:26.229210 28925 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.194@KRBTEST.COM'} at 127.24.183.194:48539
WARNING: no policy specified for kudu/127.24.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab.
Entry for principal kudu/127.24.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab.
I20260809 15:07:26.258536 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.195:0
--local_ip_for_outbound_sockets=127.24.183.195
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:38131
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:37605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:26.579106 29408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:26.579370 29408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:26.579474 29408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:26.586613 29408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:26.586711 29408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:26.586823 29408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.195
I20260809 15:07:26.596293 29408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:37605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:38131
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.195
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:26.597527 29408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:26.598776 29408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:26.611224 29416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:26.611256 29421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:26.611474 29418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:26.612174 29408 server_base.cc:1034] running on GCE node
I20260809 15:07:26.612711 29408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:26.613548 29408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:26.614748 29408 hybrid_clock.cc:648] HybridClock initialized: now 1786288046614694 us; error 48 us; skew 500 ppm
I20260809 15:07:26.615041 29408 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:26 dist-test-slave-nj21 krb5kdc[28846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288046, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:26.617986 29408 init.cc:377] Logged in from keytab as kudu/127.24.183.195@KRBTEST.COM (short username kudu)
I20260809 15:07:26.619390 29408 webserver.cc:460] Webserver started at http://127.24.183.195:38347/ using document root <none> and password file <none>
I20260809 15:07:26.620200 29408 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:26.620319 29408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:26.620620 29408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:26.623371 29408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-2/data/instance:
uuid: "820b5e95755b426589165adcf27ba861"
format_stamp: "Formatted at 2026-08-09 15:07:26 on dist-test-slave-nj21"
server_key: "c84ff5388e5831483f274835639efba9"
server_key_iv: "ddffed61a580dabbc8deb04d52a74d9c"
server_key_version: "encryptionkey@0"
I20260809 15:07:26.624128 29408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance:
uuid: "820b5e95755b426589165adcf27ba861"
format_stamp: "Formatted at 2026-08-09 15:07:26 on dist-test-slave-nj21"
server_key: "c84ff5388e5831483f274835639efba9"
server_key_iv: "ddffed61a580dabbc8deb04d52a74d9c"
server_key_version: "encryptionkey@0"
I20260809 15:07:26.629484 29408 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 15:07:26.632673 29431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:26.633886 29408 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:07:26.634032 29408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-2/wal
uuid: "820b5e95755b426589165adcf27ba861"
format_stamp: "Formatted at 2026-08-09 15:07:26 on dist-test-slave-nj21"
server_key: "c84ff5388e5831483f274835639efba9"
server_key_iv: "ddffed61a580dabbc8deb04d52a74d9c"
server_key_version: "encryptionkey@0"
I20260809 15:07:26.634212 29408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:26.644595 29408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:26.648114 29408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:26.648371 29408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:26.649232 29408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:07:26.650519 29408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:07:26.650596 29408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:26.650714 29408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:07:26.650769 29408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:26.681731 29408 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.195:41797
I20260809 15:07:26.681777 29604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.195:41797 every 8 connection(s)
I20260809 15:07:26.683055 29408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
I20260809 15:07:26.685528 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 29408
I20260809 15:07:26.685730 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance
I20260809 15:07:26.686211 25311 external_mini_cluster.cc:1428] Setting key e265df12a4721b62150d621f49b4d183
Aug 09 15:07:26 dist-test-slave-nj21 krb5kdc[28846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288046, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:26.702940 29205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:26.685675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:34085 (local address 127.24.183.254:38131)
0809 15:07:26.685896 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:26.685902 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:26.687095 (+ 1193us) server_negotiation.cc:407] Connection header received
0809 15:07:26.689046 (+ 1951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:26.689052 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:26.689161 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:26.689246 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:26.690770 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:26.691542 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:26.693083 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:26.693502 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:26.693612 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:26.697153 (+ 3541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:26.697159 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:26.697163 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:26.697198 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:26.699884 (+ 2686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:26.700778 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:26.700785 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:26.700788 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:26.700953 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:26.701404 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:26.701410 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:26.701413 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:26.701746 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:26.701881 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:26.702497 (+ 616us) server_negotiation.cc:299] Negotiation successful
0809 15:07:26.702677 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 15:07:26.703984 29607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:26.686098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:38131 (local address 127.24.183.195:34085)
0809 15:07:26.686923 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:26.686975 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:26.688785 (+ 1810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:26.689502 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:26.689510 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:26.690045 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:26.690607 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:26.690624 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:26.691713 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:26.691719 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:26.692942 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:26.692960 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:26.693722 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:26.693730 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:26.693961 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:26.694583 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:26.694606 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:26.696938 (+ 2332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:26.700012 (+ 3074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:26.700021 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:26.700038 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:26.700637 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:26.701064 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:26.701068 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:26.701070 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:26.701272 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:26.701856 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:26.701861 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:26.702175 (+ 314us) client_negotiation.cc:769] Sending connection context
0809 15:07:26.702407 (+ 232us) client_negotiation.cc:240] Negotiation successful
0809 15:07:26.702850 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":345,"threads_started":1}
I20260809 15:07:26.705312 29605 heartbeater.cc:347] Connected to a master server at 127.24.183.254:38131
I20260809 15:07:26.705593 29605 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:26.706254 29605 heartbeater.cc:511] Master 127.24.183.254:38131 requested a full tablet report, sending...
I20260809 15:07:26.707635 28925 ts_manager.cc:194] Registered new tserver with Master: 820b5e95755b426589165adcf27ba861 (127.24.183.195:41797)
I20260809 15:07:26.708496 28925 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.195@KRBTEST.COM'} at 127.24.183.195:34085
I20260809 15:07:26.713654 25311 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:07:26.721763 29201 heartbeater.cc:503] Master 127.24.183.254:38131 was elected leader, sending a full tablet report...
W20260809 15:07:26.747169 29205 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:07:26.744844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35762 (local address 127.24.183.254:38131)
0809 15:07:26.745096 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:26.745105 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:26.745491 (+ 386us) server_negotiation.cc:407] Connection header received
0809 15:07:26.745644 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:26.745648 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:26.745716 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:26.745800 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:26.746890 (+ 1090us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35762: BlockingRecv error: recv got EOF from 127.0.0.1:35762 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 15:07:26.750309 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 29011
W20260809 15:07:26.752912 25311 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 29011: Not authorized: Client connection negotiation failed: client connection to 127.24.183.193:36901: server requires authentication, but client does not have Kerberos credentials available
I20260809 15:07:26.753055 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 29011
I20260809 15:07:26.787649 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 29211
W20260809 15:07:26.790489 25311 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 29211: Not authorized: Client connection negotiation failed: client connection to 127.24.183.194:41583: server requires authentication, but client does not have Kerberos credentials available
I20260809 15:07:26.790664 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 29211
I20260809 15:07:26.810571 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 29408
W20260809 15:07:26.813419 25311 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 29408: Not authorized: Client connection negotiation failed: client connection to 127.24.183.195:41797: server requires authentication, but client does not have Kerberos credentials available
I20260809 15:07:26.813570 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 29408
I20260809 15:07:26.834136 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 28868
I20260809 15:07:26.835454 28969 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:07:26.941279 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 28868
2026-08-09T15:07:26Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3729 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:07:27 dist-test-slave-nj21 krb5kdc[29631](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:07:27 dist-test-slave-nj21 krb5kdc[29631](info): set up 2 sockets
Aug 09 15:07:27 dist-test-slave-nj21 krb5kdc[29631](info): commencing operation
krb5kdc: starting...
W20260809 15:07:28.143172 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.003s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:07:28 dist-test-slave-nj21 krb5kdc[29631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:07:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:07:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
I20260809 15:07:28.334591 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:42103
--webserver_interface=127.24.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:45877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:42103
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:28.653626 29651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:28.653934 29651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:28.654073 29651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:28.661239 29651 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:07:28.661367 29651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:28.661451 29651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:28.661528 29651 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:07:28.661597 29651 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:07:28.671389 29651 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:45877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:42103
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:42103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:28.672710 29651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:28.674046 29651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:28.685942 29658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:28.685928 29670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:28.685963 29659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:28.686689 29651 server_base.cc:1034] running on GCE node
I20260809 15:07:28.687464 29651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:28.688843 29651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:28.690047 29651 hybrid_clock.cc:648] HybridClock initialized: now 1786288048690003 us; error 34 us; skew 500 ppm
I20260809 15:07:28.690367 29651 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:28 dist-test-slave-nj21 krb5kdc[29631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288048, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:28.696672 29651 init.cc:377] Logged in from keytab as kudu/127.24.183.254@KRBTEST.COM (short username kudu)
I20260809 15:07:28.698060 29651 webserver.cc:460] Webserver started at http://127.24.183.254:41675/ using document root <none> and password file <none>
I20260809 15:07:28.698715 29651 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:28.698832 29651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:28.699136 29651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:28.702140 29651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "75f4b65964d94cb9b358af719229cf2f"
format_stamp: "Formatted at 2026-08-09 15:07:28 on dist-test-slave-nj21"
server_key: "a3df675986440f65a2c29f84a8e79b9d"
server_key_iv: "a3fb130a6f81177796a59482aa02bdc3"
server_key_version: "encryptionkey@0"
I20260809 15:07:28.702915 29651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "75f4b65964d94cb9b358af719229cf2f"
format_stamp: "Formatted at 2026-08-09 15:07:28 on dist-test-slave-nj21"
server_key: "a3df675986440f65a2c29f84a8e79b9d"
server_key_iv: "a3fb130a6f81177796a59482aa02bdc3"
server_key_version: "encryptionkey@0"
I20260809 15:07:28.707942 29651 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 15:07:28.711213 29682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:28.712484 29651 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:07:28.712638 29651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "75f4b65964d94cb9b358af719229cf2f"
format_stamp: "Formatted at 2026-08-09 15:07:28 on dist-test-slave-nj21"
server_key: "a3df675986440f65a2c29f84a8e79b9d"
server_key_iv: "a3fb130a6f81177796a59482aa02bdc3"
server_key_version: "encryptionkey@0"
I20260809 15:07:28.712812 29651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:28.734352 29651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:28.738247 29651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:28.738509 29651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:28.756357 29651 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:42103
I20260809 15:07:28.756402 29754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:42103 every 8 connection(s)
I20260809 15:07:28.757700 29651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:07:28.759433 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 29651
I20260809 15:07:28.759711 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:07:28.760538 25311 external_mini_cluster.cc:1428] Setting key 89f54d73ac6e254f88e8b5ae82cdb1b7
I20260809 15:07:28.762244 29755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 15:07:28 dist-test-slave-nj21 krb5kdc[29631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:28.775889 29755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75f4b65964d94cb9b358af719229cf2f: Bootstrap starting.
I20260809 15:07:28.779003 29755 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75f4b65964d94cb9b358af719229cf2f: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:28.780107 29755 log.cc:824] T 00000000000000000000000000000000 P 75f4b65964d94cb9b358af719229cf2f: Log is configured to *not* fsync() on all Append() calls
I20260809 15:07:28.782827 29755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75f4b65964d94cb9b358af719229cf2f: No bootstrap required, opened a new log
I20260809 15:07:28.782893 29759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:28.762447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42750 (local address 127.24.183.254:42103)
0809 15:07:28.763240 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:28.763251 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:28.763295 (+ 44us) server_negotiation.cc:407] Connection header received
0809 15:07:28.764999 (+ 1704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:28.765020 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:28.765383 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:28.765695 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:28.767571 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:28.768996 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:28.770659 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:28.771323 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:28.771441 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:28.776118 (+ 4677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:28.776125 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:28.776139 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:28.776180 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:28.779005 (+ 2825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:28.779657 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:28.779664 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:28.779699 (+ 35us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:28.779895 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:28.780257 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:28.780262 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:28.780266 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:28.780835 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:28.781128 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:28.781550 (+ 422us) server_negotiation.cc:299] Negotiation successful
0809 15:07:28.781937 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":379,"threads_started":1}
I20260809 15:07:28.786484 25311 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:07:28.790277 29755 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75f4b65964d94cb9b358af719229cf2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75f4b65964d94cb9b358af719229cf2f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 42103 } }
I20260809 15:07:28.790623 29755 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75f4b65964d94cb9b358af719229cf2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:28.790709 29755 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75f4b65964d94cb9b358af719229cf2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75f4b65964d94cb9b358af719229cf2f, State: Initialized, Role: FOLLOWER
I20260809 15:07:28.791217 29755 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75f4b65964d94cb9b358af719229cf2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75f4b65964d94cb9b358af719229cf2f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 42103 } }
I20260809 15:07:28.791450 29755 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75f4b65964d94cb9b358af719229cf2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:07:28.791572 29755 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75f4b65964d94cb9b358af719229cf2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:07:28.791761 29755 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75f4b65964d94cb9b358af719229cf2f [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:28.793812 29755 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75f4b65964d94cb9b358af719229cf2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75f4b65964d94cb9b358af719229cf2f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 42103 } }
I20260809 15:07:28.794282 29755 leader_election.cc:302] T 00000000000000000000000000000000 P 75f4b65964d94cb9b358af719229cf2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75f4b65964d94cb9b358af719229cf2f; no voters:
I20260809 15:07:28.794929 29755 leader_election.cc:288] T 00000000000000000000000000000000 P 75f4b65964d94cb9b358af719229cf2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:07:28.795013 29763 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75f4b65964d94cb9b358af719229cf2f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:07:28.796937 29755 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75f4b65964d94cb9b358af719229cf2f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:07:28.801771 29763 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75f4b65964d94cb9b358af719229cf2f [term 1 LEADER]: Becoming Leader. State: Replica: 75f4b65964d94cb9b358af719229cf2f, State: Running, Role: LEADER
I20260809 15:07:28.802358 29763 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75f4b65964d94cb9b358af719229cf2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75f4b65964d94cb9b358af719229cf2f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 42103 } }
I20260809 15:07:28.805372 29764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75f4b65964d94cb9b358af719229cf2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "75f4b65964d94cb9b358af719229cf2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75f4b65964d94cb9b358af719229cf2f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 42103 } } }
I20260809 15:07:28.805739 29764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75f4b65964d94cb9b358af719229cf2f [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:28.806236 29765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75f4b65964d94cb9b358af719229cf2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75f4b65964d94cb9b358af719229cf2f. Latest consensus state: current_term: 1 leader_uuid: "75f4b65964d94cb9b358af719229cf2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75f4b65964d94cb9b358af719229cf2f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 42103 } } }
I20260809 15:07:28.806514 29765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75f4b65964d94cb9b358af719229cf2f [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:28.819031 29780 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:07:28.825548 29780 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:07:28.835807 29780 catalog_manager.cc:1347] Generated new cluster ID: 884c2c8d1d2d48d2a8b5e91a9901bae1
I20260809 15:07:28.835934 29780 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:07:28.842598 29759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:28.833703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42760 (local address 127.24.183.254:42103)
0809 15:07:28.833916 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:28.833923 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:28.834472 (+ 549us) server_negotiation.cc:407] Connection header received
0809 15:07:28.834647 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:28.834651 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:28.834720 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:28.834810 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:28.835986 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:28.837161 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:28.838988 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:28.839634 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:28.839858 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:28.840952 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:28.840958 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:28.840963 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:07:28.841801 (+ 838us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:28.841875 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:07:28.841904 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:28.842021 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:28.842127 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 15:07:28.842227 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 15:07:28.853471 29780 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:07:28.854758 29780 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:07:28.862866 29780 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 75f4b65964d94cb9b358af719229cf2f: Generated new TSK 0
I20260809 15:07:28.863638 29780 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:07:28.871150 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 29651
I20260809 15:07:28.872403 29744 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:07:28.962698 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 29651
2026-08-09T15:07:28Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2020 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:07:29 dist-test-slave-nj21 krb5kdc[29803](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:07:29 dist-test-slave-nj21 krb5kdc[29803](info): set up 2 sockets
Aug 09 15:07:29 dist-test-slave-nj21 krb5kdc[29803](info): commencing operation
krb5kdc: starting...
W20260809 15:07:30.161809 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.001s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:07:30 dist-test-slave-nj21 krb5kdc[29803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:07:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:07:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
I20260809 15:07:30.369127 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:33419
--webserver_interface=127.24.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:43441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:33419
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:30.695585 29824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:30.695930 29824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:30.696043 29824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:30.703624 29824 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:07:30.703771 29824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:30.703835 29824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:30.703899 29824 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:07:30.703951 29824 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:07:30.714252 29824 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:43441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:33419
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:33419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:30.715641 29824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:30.717078 29824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:30.728880 29835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:30.728951 29833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:30.728905 29837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:30.729630 29824 server_base.cc:1034] running on GCE node
I20260809 15:07:30.730418 29824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:30.731837 29824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:30.733065 29824 hybrid_clock.cc:648] HybridClock initialized: now 1786288050733001 us; error 37 us; skew 500 ppm
I20260809 15:07:30.733496 29824 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:30 dist-test-slave-nj21 krb5kdc[29803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288050, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:30.736497 29824 init.cc:377] Logged in from keytab as kudu/127.24.183.254@KRBTEST.COM (short username kudu)
I20260809 15:07:30.737983 29824 webserver.cc:460] Webserver started at http://127.24.183.254:41105/ using document root <none> and password file <none>
I20260809 15:07:30.738639 29824 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:30.738755 29824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:30.739048 29824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:30.742002 29824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "c786285516394200a1b11775e7fee798"
format_stamp: "Formatted at 2026-08-09 15:07:30 on dist-test-slave-nj21"
server_key: "c1659b2cdfeece5517bbe826ae4f7634"
server_key_iv: "97fefbbfbaad6615fda39f438a3f97e3"
server_key_version: "encryptionkey@0"
I20260809 15:07:30.742734 29824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "c786285516394200a1b11775e7fee798"
format_stamp: "Formatted at 2026-08-09 15:07:30 on dist-test-slave-nj21"
server_key: "c1659b2cdfeece5517bbe826ae4f7634"
server_key_iv: "97fefbbfbaad6615fda39f438a3f97e3"
server_key_version: "encryptionkey@0"
I20260809 15:07:30.747807 29824 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260809 15:07:30.751055 29849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:30.752327 29824 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:07:30.752485 29824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "c786285516394200a1b11775e7fee798"
format_stamp: "Formatted at 2026-08-09 15:07:30 on dist-test-slave-nj21"
server_key: "c1659b2cdfeece5517bbe826ae4f7634"
server_key_iv: "97fefbbfbaad6615fda39f438a3f97e3"
server_key_version: "encryptionkey@0"
I20260809 15:07:30.752653 29824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:30.782308 29824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:30.786255 29824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:30.786530 29824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:30.804128 29824 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:33419
I20260809 15:07:30.804153 29921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:33419 every 8 connection(s)
I20260809 15:07:30.805469 29824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:07:30.808677 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 29824
I20260809 15:07:30.808877 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:07:30.809569 29923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:30.809751 25311 external_mini_cluster.cc:1428] Setting key eb4fb106f5c4e47f3d91c20c84655c1e
Aug 09 15:07:30 dist-test-slave-nj21 krb5kdc[29803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:30.824302 29923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c786285516394200a1b11775e7fee798: Bootstrap starting.
I20260809 15:07:30.827881 29923 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c786285516394200a1b11775e7fee798: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:30.829048 29923 log.cc:824] T 00000000000000000000000000000000 P c786285516394200a1b11775e7fee798: Log is configured to *not* fsync() on all Append() calls
I20260809 15:07:30.832568 29923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c786285516394200a1b11775e7fee798: No bootstrap required, opened a new log
I20260809 15:07:30.833317 29930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:30.811841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54454 (local address 127.24.183.254:33419)
0809 15:07:30.812644 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:30.812655 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:30.812698 (+ 43us) server_negotiation.cc:407] Connection header received
0809 15:07:30.814366 (+ 1668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:30.814389 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:30.814757 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:30.815066 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:30.816859 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:30.818145 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:30.819909 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:30.820628 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:30.820764 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:30.825605 (+ 4841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:30.825614 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:30.825630 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:30.825669 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:30.828822 (+ 3153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:30.829680 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:30.829690 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:30.829700 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:30.829941 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:30.830319 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:30.830324 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:30.830327 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:30.830918 (+ 591us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:30.831145 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:30.831804 (+ 659us) server_negotiation.cc:299] Negotiation successful
0809 15:07:30.832216 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":361,"threads_started":1}
I20260809 15:07:30.840758 29923 raft_consensus.cc:348] T 00000000000000000000000000000000 P c786285516394200a1b11775e7fee798 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c786285516394200a1b11775e7fee798" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 33419 } }
I20260809 15:07:30.841120 29923 raft_consensus.cc:374] T 00000000000000000000000000000000 P c786285516394200a1b11775e7fee798 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:30.841230 29923 raft_consensus.cc:729] T 00000000000000000000000000000000 P c786285516394200a1b11775e7fee798 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c786285516394200a1b11775e7fee798, State: Initialized, Role: FOLLOWER
I20260809 15:07:30.841795 29923 consensus_queue.cc:260] T 00000000000000000000000000000000 P c786285516394200a1b11775e7fee798 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c786285516394200a1b11775e7fee798" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 33419 } }
I20260809 15:07:30.842017 29923 raft_consensus.cc:388] T 00000000000000000000000000000000 P c786285516394200a1b11775e7fee798 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:07:30.842145 29923 raft_consensus.cc:482] T 00000000000000000000000000000000 P c786285516394200a1b11775e7fee798 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:07:30.842300 29923 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c786285516394200a1b11775e7fee798 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:30.844692 29923 raft_consensus.cc:504] T 00000000000000000000000000000000 P c786285516394200a1b11775e7fee798 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c786285516394200a1b11775e7fee798" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 33419 } }
I20260809 15:07:30.845184 29923 leader_election.cc:302] T 00000000000000000000000000000000 P c786285516394200a1b11775e7fee798 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c786285516394200a1b11775e7fee798; no voters:
I20260809 15:07:30.845762 29923 leader_election.cc:288] T 00000000000000000000000000000000 P c786285516394200a1b11775e7fee798 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:07:30.845885 29934 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c786285516394200a1b11775e7fee798 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:07:30.846545 29934 raft_consensus.cc:686] T 00000000000000000000000000000000 P c786285516394200a1b11775e7fee798 [term 1 LEADER]: Becoming Leader. State: Replica: c786285516394200a1b11775e7fee798, State: Running, Role: LEADER
I20260809 15:07:30.847043 29934 consensus_queue.cc:237] T 00000000000000000000000000000000 P c786285516394200a1b11775e7fee798 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c786285516394200a1b11775e7fee798" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 33419 } }
I20260809 15:07:30.847919 29923 sys_catalog.cc:564] T 00000000000000000000000000000000 P c786285516394200a1b11775e7fee798 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:07:30.850047 29938 sys_catalog.cc:455] T 00000000000000000000000000000000 P c786285516394200a1b11775e7fee798 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c786285516394200a1b11775e7fee798" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c786285516394200a1b11775e7fee798" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 33419 } } }
I20260809 15:07:30.850014 29940 sys_catalog.cc:455] T 00000000000000000000000000000000 P c786285516394200a1b11775e7fee798 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c786285516394200a1b11775e7fee798. Latest consensus state: current_term: 1 leader_uuid: "c786285516394200a1b11775e7fee798" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c786285516394200a1b11775e7fee798" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 33419 } } }
I20260809 15:07:30.850445 29938 sys_catalog.cc:458] T 00000000000000000000000000000000 P c786285516394200a1b11775e7fee798 [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:30.850448 29940 sys_catalog.cc:458] T 00000000000000000000000000000000 P c786285516394200a1b11775e7fee798 [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:30.855013 29943 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:07:30.860426 29943 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:07:30.868744 29943 catalog_manager.cc:1347] Generated new cluster ID: dd0e49872fe24b11ba5dee2df5832991
I20260809 15:07:30.868850 29943 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:07:30.878788 29943 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:07:30.880113 29943 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:07:30.888213 29943 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c786285516394200a1b11775e7fee798: Generated new TSK 0
I20260809 15:07:30.888989 29943 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab.
Entry for principal kudu/127.24.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab.
I20260809 15:07:30.946808 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.193:0
--local_ip_for_outbound_sockets=127.24.183.193
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:33419
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:43441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:31.305364 29970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:31.305630 29970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:31.305757 29970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:31.313196 29970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:31.313310 29970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:31.313457 29970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.193
I20260809 15:07:31.323278 29970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:43441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:33419
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.193
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:31.324769 29970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:31.326108 29970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:31.339563 29977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:31.339519 29976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:31.339587 29981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:31.340340 29970 server_base.cc:1034] running on GCE node
I20260809 15:07:31.340932 29970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:31.341847 29970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:31.343027 29970 hybrid_clock.cc:648] HybridClock initialized: now 1786288051342974 us; error 45 us; skew 500 ppm
I20260809 15:07:31.343349 29970 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:31 dist-test-slave-nj21 krb5kdc[29803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288051, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:31.346453 29970 init.cc:377] Logged in from keytab as kudu/127.24.183.193@KRBTEST.COM (short username kudu)
I20260809 15:07:31.348043 29970 webserver.cc:460] Webserver started at http://127.24.183.193:33785/ using document root <none> and password file <none>
I20260809 15:07:31.348821 29970 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:31.348956 29970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:31.349270 29970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:31.352520 29970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-0/data/instance:
uuid: "40ed40aa011c404080e68d033ce6a392"
format_stamp: "Formatted at 2026-08-09 15:07:31 on dist-test-slave-nj21"
server_key: "49967d7a8dd49f94e2d515a0812c6e0a"
server_key_iv: "a490b77253f5b5a7e7b3a1db750cc30c"
server_key_version: "encryptionkey@0"
I20260809 15:07:31.353307 29970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance:
uuid: "40ed40aa011c404080e68d033ce6a392"
format_stamp: "Formatted at 2026-08-09 15:07:31 on dist-test-slave-nj21"
server_key: "49967d7a8dd49f94e2d515a0812c6e0a"
server_key_iv: "a490b77253f5b5a7e7b3a1db750cc30c"
server_key_version: "encryptionkey@0"
I20260809 15:07:31.358601 29970 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260809 15:07:31.362170 29990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:31.363483 29970 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 15:07:31.363642 29970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-0/wal
uuid: "40ed40aa011c404080e68d033ce6a392"
format_stamp: "Formatted at 2026-08-09 15:07:31 on dist-test-slave-nj21"
server_key: "49967d7a8dd49f94e2d515a0812c6e0a"
server_key_iv: "a490b77253f5b5a7e7b3a1db750cc30c"
server_key_version: "encryptionkey@0"
I20260809 15:07:31.363870 29970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:31.376240 29970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:31.380069 29970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:31.380349 29970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:31.381311 29970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:07:31.382656 29970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:07:31.382747 29970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:31.382867 29970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:07:31.382923 29970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:31.414232 29970 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.193:42409
I20260809 15:07:31.414268 30168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.193:42409 every 8 connection(s)
I20260809 15:07:31.415531 29970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
I20260809 15:07:31.421165 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 29970
I20260809 15:07:31.421360 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance
I20260809 15:07:31.421983 25311 external_mini_cluster.cc:1428] Setting key 63bc5750a7feb5bec8ff3f8aab064420
Aug 09 15:07:31 dist-test-slave-nj21 krb5kdc[29803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288051, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:31.436121 30173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:31.418122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:42713 (local address 127.24.183.254:33419)
0809 15:07:31.418582 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:31.418588 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:31.419590 (+ 1002us) server_negotiation.cc:407] Connection header received
0809 15:07:31.421459 (+ 1869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:31.421465 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:31.421563 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:31.421661 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:31.423391 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:31.424320 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:31.425913 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:31.426505 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:31.426630 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:31.430367 (+ 3737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:31.430373 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:31.430376 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:31.430412 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:31.432996 (+ 2584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:31.433714 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:31.433720 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:31.433724 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:31.433876 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:31.434268 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:31.434272 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:31.434275 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:31.434620 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:31.434763 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:31.435472 (+ 709us) server_negotiation.cc:299] Negotiation successful
0809 15:07:31.435751 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":213,"threads_started":1}
I20260809 15:07:31.436969 30174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:31.418702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:33419 (local address 127.24.183.193:42713)
0809 15:07:31.419425 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:31.419470 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:31.421176 (+ 1706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:31.421915 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:31.421924 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:31.422466 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:31.423200 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:31.423225 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:31.424475 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:31.424479 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:31.425748 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:31.425762 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:31.426696 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:31.426701 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:31.426913 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:31.427575 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:31.427599 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:31.430117 (+ 2518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:31.433127 (+ 3010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:31.433134 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:31.433148 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:31.433593 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:31.433970 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:31.433973 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:31.433975 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:31.434167 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:31.434722 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:31.434728 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:31.435057 (+ 329us) client_negotiation.cc:769] Sending connection context
0809 15:07:31.435323 (+ 266us) client_negotiation.cc:240] Negotiation successful
0809 15:07:31.435800 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":274,"threads_started":1}
I20260809 15:07:31.438649 30170 heartbeater.cc:347] Connected to a master server at 127.24.183.254:33419
I20260809 15:07:31.439008 30170 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:31.440018 30170 heartbeater.cc:511] Master 127.24.183.254:33419 requested a full tablet report, sending...
I20260809 15:07:31.442417 29878 ts_manager.cc:194] Registered new tserver with Master: 40ed40aa011c404080e68d033ce6a392 (127.24.183.193:42409)
I20260809 15:07:31.443969 29878 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.193@KRBTEST.COM'} at 127.24.183.193:42713
WARNING: no policy specified for kudu/127.24.183.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab.
Entry for principal kudu/127.24.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab.
I20260809 15:07:31.470474 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.194:0
--local_ip_for_outbound_sockets=127.24.183.194
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:33419
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:43441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:31.800836 30181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:31.801119 30181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:31.801263 30181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:31.808729 30181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:31.808866 30181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:31.809026 30181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.194
I20260809 15:07:31.818866 30181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:43441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:33419
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.194
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:31.820240 30181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:31.821595 30181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:31.834417 30193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:31.834383 30195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:31.834444 30197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:31.835176 30181 server_base.cc:1034] running on GCE node
I20260809 15:07:31.835875 30181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:31.836817 30181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:31.838018 30181 hybrid_clock.cc:648] HybridClock initialized: now 1786288051837972 us; error 45 us; skew 500 ppm
I20260809 15:07:31.838325 30181 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:31 dist-test-slave-nj21 krb5kdc[29803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288051, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:31.841394 30181 init.cc:377] Logged in from keytab as kudu/127.24.183.194@KRBTEST.COM (short username kudu)
I20260809 15:07:31.842870 30181 webserver.cc:460] Webserver started at http://127.24.183.194:39049/ using document root <none> and password file <none>
I20260809 15:07:31.843515 30181 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:31.843617 30181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:31.843937 30181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:31.846810 30181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-1/data/instance:
uuid: "63d63384236745debf34d22c34f34588"
format_stamp: "Formatted at 2026-08-09 15:07:31 on dist-test-slave-nj21"
server_key: "210ef6335f4ef0d204c7ca2d406c935b"
server_key_iv: "4383c68eab99fb479802378eb40d1a6b"
server_key_version: "encryptionkey@0"
I20260809 15:07:31.847550 30181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance:
uuid: "63d63384236745debf34d22c34f34588"
format_stamp: "Formatted at 2026-08-09 15:07:31 on dist-test-slave-nj21"
server_key: "210ef6335f4ef0d204c7ca2d406c935b"
server_key_iv: "4383c68eab99fb479802378eb40d1a6b"
server_key_version: "encryptionkey@0"
I20260809 15:07:31.852847 30181 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 15:07:31.856297 30209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:31.857636 30181 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:07:31.857766 30181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-1/wal
uuid: "63d63384236745debf34d22c34f34588"
format_stamp: "Formatted at 2026-08-09 15:07:31 on dist-test-slave-nj21"
server_key: "210ef6335f4ef0d204c7ca2d406c935b"
server_key_iv: "4383c68eab99fb479802378eb40d1a6b"
server_key_version: "encryptionkey@0"
I20260809 15:07:31.857939 30181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:31.874927 30181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:31.878755 30181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:31.879015 30181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:31.879987 30181 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:07:31.881307 30181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:07:31.881381 30181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:31.881490 30181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:07:31.881533 30181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:31.913260 30181 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.194:44421
I20260809 15:07:31.913277 30380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.194:44421 every 8 connection(s)
I20260809 15:07:31.914562 30181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
I20260809 15:07:31.919574 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 30181
I20260809 15:07:31.919790 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance
I20260809 15:07:31.920375 25311 external_mini_cluster.cc:1428] Setting key 0b24dc197564daf82eede0076a46b971
Aug 09 15:07:31 dist-test-slave-nj21 krb5kdc[29803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288051, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:31.936058 30173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:31.917105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:55387 (local address 127.24.183.254:33419)
0809 15:07:31.917288 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:31.917292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:31.918468 (+ 1176us) server_negotiation.cc:407] Connection header received
0809 15:07:31.920779 (+ 2311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:31.920784 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:31.920862 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:31.920950 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:31.922718 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:31.923432 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:31.925628 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:31.926121 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:31.926217 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:31.930295 (+ 4078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:31.930300 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:31.930303 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:31.930337 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:31.933032 (+ 2695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:31.933752 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:31.933757 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:31.933759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:31.933906 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:31.934330 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:31.934334 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:31.934336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:31.934609 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:31.934745 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:31.935429 (+ 684us) server_negotiation.cc:299] Negotiation successful
0809 15:07:31.935686 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 15:07:31.937072 30385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:31.917595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:33419 (local address 127.24.183.194:55387)
0809 15:07:31.918308 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:31.918351 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:31.920463 (+ 2112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:31.921184 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:31.921193 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:31.921835 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:31.922548 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:31.922568 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:31.923596 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:31.923601 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:31.925474 (+ 1873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:31.925488 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:31.926346 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:31.926353 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:31.926627 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:31.927312 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:31.927343 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:31.930046 (+ 2703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:31.933179 (+ 3133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:31.933186 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:31.933200 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:31.933625 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:31.934011 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:31.934015 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:31.934017 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:31.934202 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:31.934730 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:31.934738 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:31.935074 (+ 336us) client_negotiation.cc:769] Sending connection context
0809 15:07:31.935313 (+ 239us) client_negotiation.cc:240] Negotiation successful
0809 15:07:31.935887 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":279,"threads_started":1}
I20260809 15:07:31.938597 30382 heartbeater.cc:347] Connected to a master server at 127.24.183.254:33419
I20260809 15:07:31.938892 30382 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:31.939704 30382 heartbeater.cc:511] Master 127.24.183.254:33419 requested a full tablet report, sending...
I20260809 15:07:31.941201 29878 ts_manager.cc:194] Registered new tserver with Master: 63d63384236745debf34d22c34f34588 (127.24.183.194:44421)
I20260809 15:07:31.942021 29878 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.194@KRBTEST.COM'} at 127.24.183.194:55387
WARNING: no policy specified for kudu/127.24.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab.
Entry for principal kudu/127.24.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab.
I20260809 15:07:31.970124 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.195:0
--local_ip_for_outbound_sockets=127.24.183.195
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:33419
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:43441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:32.299487 30388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:32.299809 30388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:32.299930 30388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:32.307231 30388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:32.307331 30388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:32.307453 30388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.195
I20260809 15:07:32.317209 30388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:43441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:33419
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.195
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:32.318539 30388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:32.319819 30388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:32.332645 30396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:32.332612 30399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:32.332695 30395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:32.333824 30388 server_base.cc:1034] running on GCE node
I20260809 15:07:32.334374 30388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:32.335296 30388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:32.336511 30388 hybrid_clock.cc:648] HybridClock initialized: now 1786288052336430 us; error 73 us; skew 500 ppm
I20260809 15:07:32.336788 30388 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:32 dist-test-slave-nj21 krb5kdc[29803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288052, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:32.339932 30388 init.cc:377] Logged in from keytab as kudu/127.24.183.195@KRBTEST.COM (short username kudu)
I20260809 15:07:32.341274 30388 webserver.cc:460] Webserver started at http://127.24.183.195:35335/ using document root <none> and password file <none>
I20260809 15:07:32.341905 30388 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:32.342007 30388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:32.342259 30388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:32.345137 30388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-2/data/instance:
uuid: "c814cd0623884c89a7a2608b2fb68f89"
format_stamp: "Formatted at 2026-08-09 15:07:32 on dist-test-slave-nj21"
server_key: "c98ad2ddf5263c859d958d2f549a17d7"
server_key_iv: "d6750a23b975c55704d7b65f0c85d293"
server_key_version: "encryptionkey@0"
I20260809 15:07:32.345868 30388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance:
uuid: "c814cd0623884c89a7a2608b2fb68f89"
format_stamp: "Formatted at 2026-08-09 15:07:32 on dist-test-slave-nj21"
server_key: "c98ad2ddf5263c859d958d2f549a17d7"
server_key_iv: "d6750a23b975c55704d7b65f0c85d293"
server_key_version: "encryptionkey@0"
I20260809 15:07:32.351042 30388 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 15:07:32.354314 30410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:32.355621 30388 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:07:32.355786 30388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-2/wal
uuid: "c814cd0623884c89a7a2608b2fb68f89"
format_stamp: "Formatted at 2026-08-09 15:07:32 on dist-test-slave-nj21"
server_key: "c98ad2ddf5263c859d958d2f549a17d7"
server_key_iv: "d6750a23b975c55704d7b65f0c85d293"
server_key_version: "encryptionkey@0"
I20260809 15:07:32.355962 30388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:32.377919 30388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:32.382033 30388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:32.382318 30388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:32.383277 30388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:07:32.384661 30388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:07:32.384754 30388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:32.384878 30388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:07:32.384934 30388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:32.414920 30388 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.195:37613
I20260809 15:07:32.414990 30575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.195:37613 every 8 connection(s)
I20260809 15:07:32.416244 30388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
I20260809 15:07:32.419234 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 30388
I20260809 15:07:32.419418 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance
I20260809 15:07:32.419967 25311 external_mini_cluster.cc:1428] Setting key e3a0f8f7df0c16afb7bfa7057eb03dfd
Aug 09 15:07:32 dist-test-slave-nj21 krb5kdc[29803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288052, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:32.435612 30173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:32.419177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:47907 (local address 127.24.183.254:33419)
0809 15:07:32.419408 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:32.419414 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:32.420737 (+ 1323us) server_negotiation.cc:407] Connection header received
0809 15:07:32.422668 (+ 1931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:32.422674 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:32.422780 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:32.422866 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:32.424319 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:32.425066 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:32.426536 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:32.426952 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:32.427060 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:32.430279 (+ 3219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:32.430284 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:32.430288 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:32.430320 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:32.432737 (+ 2417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:32.433414 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:32.433418 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:32.433420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:32.433545 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:32.433957 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:32.433960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:32.433962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:32.434234 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:32.434378 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:32.435038 (+ 660us) server_negotiation.cc:299] Negotiation successful
0809 15:07:32.435261 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260809 15:07:32.436502 30580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:32.419737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:33419 (local address 127.24.183.195:47907)
0809 15:07:32.420575 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:32.420616 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:32.422417 (+ 1801us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:32.423093 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:32.423101 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:32.423579 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:32.424164 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:32.424181 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:32.425213 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:32.425217 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:32.426402 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:32.426414 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:32.427164 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:32.427171 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:32.427382 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:32.427988 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:32.428009 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:32.430087 (+ 2078us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:32.432882 (+ 2795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:32.432891 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:32.432905 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:32.433292 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:32.433647 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:32.433652 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:32.433656 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:32.433845 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:32.434357 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:32.434365 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:32.434668 (+ 303us) client_negotiation.cc:769] Sending connection context
0809 15:07:32.434900 (+ 232us) client_negotiation.cc:240] Negotiation successful
0809 15:07:32.435391 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":310,"threads_started":1}
I20260809 15:07:32.437712 30577 heartbeater.cc:347] Connected to a master server at 127.24.183.254:33419
I20260809 15:07:32.437983 30577 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:32.438642 30577 heartbeater.cc:511] Master 127.24.183.254:33419 requested a full tablet report, sending...
I20260809 15:07:32.440050 29878 ts_manager.cc:194] Registered new tserver with Master: c814cd0623884c89a7a2608b2fb68f89 (127.24.183.195:37613)
I20260809 15:07:32.440862 29878 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.195@KRBTEST.COM'} at 127.24.183.195:47907
I20260809 15:07:32.446877 30170 heartbeater.cc:503] Master 127.24.183.254:33419 was elected leader, sending a full tablet report...
I20260809 15:07:32.447122 25311 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 15:07:32 dist-test-slave-nj21 krb5kdc[29803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288052, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 15:07:32 dist-test-slave-nj21 krb5kdc[29803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288052, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:32.497977 30173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:32.484400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54468 (local address 127.24.183.254:33419)
0809 15:07:32.484600 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:32.484608 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:32.485012 (+ 404us) server_negotiation.cc:407] Connection header received
0809 15:07:32.485199 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:32.485204 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:32.485277 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:32.485377 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:32.486805 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:32.487572 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:32.488843 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:32.489279 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:32.489398 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:32.493064 (+ 3666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:32.493069 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:32.493072 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:32.493107 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:32.495404 (+ 2297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:32.496060 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:32.496065 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:32.496067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:32.496206 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:32.496576 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:32.496580 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:32.496582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:32.496909 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:32.497149 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:32.497446 (+ 297us) server_negotiation.cc:299] Negotiation successful
0809 15:07:32.497666 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260809 15:07:32.498589 29878 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:54468
I20260809 15:07:32.500849 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 29970
Aug 09 15:07:32 dist-test-slave-nj21 krb5kdc[29803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288052, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.24.183.193@KRBTEST.COM
I20260809 15:07:32.515177 30592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:32.501746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43938 (local address 127.24.183.193:42409)
0809 15:07:32.502364 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:32.502371 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:32.502396 (+ 25us) server_negotiation.cc:407] Connection header received
0809 15:07:32.502454 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:32.502460 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:32.502677 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:32.502799 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:32.503918 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:32.504931 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:32.506036 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:32.506566 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:32.506657 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:32.510242 (+ 3585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:32.510254 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:32.510268 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:32.510305 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:32.512985 (+ 2680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:32.513512 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:32.513518 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:32.513520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:32.513641 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:32.513903 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:32.513906 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:32.513907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:32.514235 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:32.514451 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:32.514687 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 15:07:32.514888 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":315,"threads_started":1}
W20260809 15:07:32.516461 30162 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:43938
W20260809 15:07:32.517468 25311 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 29970: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:07:32.517663 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 29970
I20260809 15:07:32.542284 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 30181
Aug 09 15:07:32 dist-test-slave-nj21 krb5kdc[29803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288052, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.24.183.194@KRBTEST.COM
I20260809 15:07:32.557969 30593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:32.543167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34496 (local address 127.24.183.194:44421)
0809 15:07:32.543723 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:32.543733 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:32.543763 (+ 30us) server_negotiation.cc:407] Connection header received
0809 15:07:32.543836 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:32.543844 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:32.544066 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:32.544198 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:32.545394 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:32.546395 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:32.547577 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:32.548303 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:32.548414 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:32.552392 (+ 3978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:32.552403 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:32.552418 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:32.552452 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:32.555070 (+ 2618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:32.555893 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:32.555902 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:32.555904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:32.556050 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:32.556392 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:32.556395 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:32.556397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:32.556874 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:32.557125 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:32.557415 (+ 290us) server_negotiation.cc:299] Negotiation successful
0809 15:07:32.557642 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":273,"threads_started":1}
W20260809 15:07:32.559461 30370 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:34496
W20260809 15:07:32.560684 25311 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 30181: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:07:32.560868 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 30181
I20260809 15:07:32.584837 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 30388
Aug 09 15:07:32 dist-test-slave-nj21 krb5kdc[29803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288052, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.24.183.195@KRBTEST.COM
I20260809 15:07:32.600811 30595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:32.585920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54186 (local address 127.24.183.195:37613)
0809 15:07:32.586573 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:32.586580 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:32.586608 (+ 28us) server_negotiation.cc:407] Connection header received
0809 15:07:32.586670 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:32.586677 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:32.586891 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:32.587037 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:32.588539 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:32.589605 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:32.590825 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:32.591387 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:32.591501 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:32.595761 (+ 4260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:32.595772 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:32.595788 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:32.595826 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:32.598397 (+ 2571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:32.599007 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:32.599013 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:32.599015 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:32.599136 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:32.599406 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:32.599408 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:32.599410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:32.599777 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:32.600004 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:32.600242 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 15:07:32.600459 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":325,"threads_started":1}
W20260809 15:07:32.602159 30569 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:54186
W20260809 15:07:32.603188 25311 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 30388: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:07:32.603358 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 30388
I20260809 15:07:32.623211 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 29824
I20260809 15:07:32.624478 29917 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:07:32.731303 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 29824
2026-08-09T15:07:32Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3773 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:07:32 dist-test-slave-nj21 krb5kdc[30599](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:07:32 dist-test-slave-nj21 krb5kdc[30599](info): set up 2 sockets
Aug 09 15:07:32 dist-test-slave-nj21 krb5kdc[30599](info): commencing operation
krb5kdc: starting...
W20260809 15:07:33.934283 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:07:34 dist-test-slave-nj21 krb5kdc[30599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288054, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:07:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:07:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
I20260809 15:07:34.136953 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:41027
--webserver_interface=127.24.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:34107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:41027
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:34.473366 30618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:34.473644 30618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:34.473775 30618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:34.482834 30618 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:07:34.482960 30618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:34.483043 30618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:34.483124 30618 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:07:34.483181 30618 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:07:34.493288 30618 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:34107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:41027
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:41027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:34.494648 30618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:34.496002 30618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:34.508109 30626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:34.508232 30635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:34.508323 30627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:34.509413 30618 server_base.cc:1034] running on GCE node
I20260809 15:07:34.510207 30618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:34.511647 30618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:34.512914 30618 hybrid_clock.cc:648] HybridClock initialized: now 1786288054512863 us; error 37 us; skew 500 ppm
I20260809 15:07:34.513298 30618 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:34 dist-test-slave-nj21 krb5kdc[30599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288054, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:34.516312 30618 init.cc:377] Logged in from keytab as kudu/127.24.183.254@KRBTEST.COM (short username kudu)
I20260809 15:07:34.517722 30618 webserver.cc:460] Webserver started at http://127.24.183.254:35389/ using document root <none> and password file <none>
I20260809 15:07:34.518414 30618 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:34.518527 30618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:34.518823 30618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:34.521804 30618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "28fc178956ef46eba99b65e84f2a506f"
format_stamp: "Formatted at 2026-08-09 15:07:34 on dist-test-slave-nj21"
server_key: "892b5d3ecd7baace248d432f3afb0e7d"
server_key_iv: "11277703dc3d1c681c85da0e853edc81"
server_key_version: "encryptionkey@0"
I20260809 15:07:34.522557 30618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "28fc178956ef46eba99b65e84f2a506f"
format_stamp: "Formatted at 2026-08-09 15:07:34 on dist-test-slave-nj21"
server_key: "892b5d3ecd7baace248d432f3afb0e7d"
server_key_iv: "11277703dc3d1c681c85da0e853edc81"
server_key_version: "encryptionkey@0"
I20260809 15:07:34.527562 30618 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 15:07:34.530961 30643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:34.532322 30618 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:07:34.532490 30618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "28fc178956ef46eba99b65e84f2a506f"
format_stamp: "Formatted at 2026-08-09 15:07:34 on dist-test-slave-nj21"
server_key: "892b5d3ecd7baace248d432f3afb0e7d"
server_key_iv: "11277703dc3d1c681c85da0e853edc81"
server_key_version: "encryptionkey@0"
I20260809 15:07:34.532665 30618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:34.565168 30618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:34.569139 30618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:34.569427 30618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:34.587150 30618 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:41027
I20260809 15:07:34.587174 30718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:41027 every 8 connection(s)
I20260809 15:07:34.588519 30618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:07:34.592360 30720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:34.597893 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 30618
I20260809 15:07:34.598098 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:07:34.598739 25311 external_mini_cluster.cc:1428] Setting key a3017714e75180e40ea7690510d12457
I20260809 15:07:34.603351 30720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28fc178956ef46eba99b65e84f2a506f: Bootstrap starting.
I20260809 15:07:34.608125 30720 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28fc178956ef46eba99b65e84f2a506f: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:34.609464 30720 log.cc:824] T 00000000000000000000000000000000 P 28fc178956ef46eba99b65e84f2a506f: Log is configured to *not* fsync() on all Append() calls
I20260809 15:07:34.612794 30720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28fc178956ef46eba99b65e84f2a506f: No bootstrap required, opened a new log
Aug 09 15:07:34 dist-test-slave-nj21 krb5kdc[30599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288054, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:34.622455 30720 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28fc178956ef46eba99b65e84f2a506f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28fc178956ef46eba99b65e84f2a506f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 41027 } }
I20260809 15:07:34.622937 30720 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28fc178956ef46eba99b65e84f2a506f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:34.623076 30720 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28fc178956ef46eba99b65e84f2a506f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28fc178956ef46eba99b65e84f2a506f, State: Initialized, Role: FOLLOWER
I20260809 15:07:34.623838 30720 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28fc178956ef46eba99b65e84f2a506f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28fc178956ef46eba99b65e84f2a506f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 41027 } }
I20260809 15:07:34.624145 30720 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28fc178956ef46eba99b65e84f2a506f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:07:34.624289 30720 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28fc178956ef46eba99b65e84f2a506f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:07:34.624493 30720 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28fc178956ef46eba99b65e84f2a506f [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:34.625398 30725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:34.600772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47614 (local address 127.24.183.254:41027)
0809 15:07:34.601609 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:34.601624 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:34.601671 (+ 47us) server_negotiation.cc:407] Connection header received
0809 15:07:34.604066 (+ 2395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:34.604098 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:34.604584 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:34.605897 (+ 1313us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:34.607750 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:34.609805 (+ 2055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:34.611418 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:34.612471 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:34.612656 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:34.616636 (+ 3980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:34.616647 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:34.616665 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:34.616711 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:34.620689 (+ 3978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:34.621303 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:34.621312 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:34.621324 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:34.621558 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:34.621892 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:34.621897 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:34.621900 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:34.622621 (+ 721us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:34.622916 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:34.623280 (+ 364us) server_negotiation.cc:299] Negotiation successful
0809 15:07:34.623941 (+ 661us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":348,"threads_started":1}
I20260809 15:07:34.627142 30720 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28fc178956ef46eba99b65e84f2a506f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28fc178956ef46eba99b65e84f2a506f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 41027 } }
I20260809 15:07:34.627628 30720 leader_election.cc:302] T 00000000000000000000000000000000 P 28fc178956ef46eba99b65e84f2a506f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28fc178956ef46eba99b65e84f2a506f; no voters:
I20260809 15:07:34.628297 30720 leader_election.cc:288] T 00000000000000000000000000000000 P 28fc178956ef46eba99b65e84f2a506f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:07:34.628379 30727 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28fc178956ef46eba99b65e84f2a506f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:07:34.629101 30727 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28fc178956ef46eba99b65e84f2a506f [term 1 LEADER]: Becoming Leader. State: Replica: 28fc178956ef46eba99b65e84f2a506f, State: Running, Role: LEADER
I20260809 15:07:34.629650 30727 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28fc178956ef46eba99b65e84f2a506f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28fc178956ef46eba99b65e84f2a506f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 41027 } }
I20260809 15:07:34.630328 30720 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28fc178956ef46eba99b65e84f2a506f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:07:34.632239 30728 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28fc178956ef46eba99b65e84f2a506f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28fc178956ef46eba99b65e84f2a506f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28fc178956ef46eba99b65e84f2a506f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 41027 } } }
I20260809 15:07:34.632210 30729 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28fc178956ef46eba99b65e84f2a506f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28fc178956ef46eba99b65e84f2a506f. Latest consensus state: current_term: 1 leader_uuid: "28fc178956ef46eba99b65e84f2a506f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28fc178956ef46eba99b65e84f2a506f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 41027 } } }
I20260809 15:07:34.632958 30729 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28fc178956ef46eba99b65e84f2a506f [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:34.632658 30728 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28fc178956ef46eba99b65e84f2a506f [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:34.634971 30737 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:07:34.641181 30737 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:07:34.650152 30737 catalog_manager.cc:1347] Generated new cluster ID: 8ef8d2d7260d48b585944caa50eefdb5
I20260809 15:07:34.650247 30737 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:07:34.660840 30737 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:07:34.662082 30737 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:07:34.671856 30737 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28fc178956ef46eba99b65e84f2a506f: Generated new TSK 0
I20260809 15:07:34.672719 30737 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab.
Entry for principal kudu/127.24.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab.
I20260809 15:07:34.739164 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.193:0
--local_ip_for_outbound_sockets=127.24.183.193
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:41027
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:34107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:35.092415 30758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:35.092674 30758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:35.092806 30758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:35.099879 30758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:35.099982 30758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:35.100142 30758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.193
I20260809 15:07:35.109685 30758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:34107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:41027
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.193
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:35.110984 30758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:35.112394 30758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:35.124802 30767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:35.124879 30766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:35.125053 30771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:35.125914 30758 server_base.cc:1034] running on GCE node
I20260809 15:07:35.126529 30758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:35.127386 30758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:35.128576 30758 hybrid_clock.cc:648] HybridClock initialized: now 1786288055128535 us; error 32 us; skew 500 ppm
I20260809 15:07:35.128875 30758 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:35 dist-test-slave-nj21 krb5kdc[30599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288055, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:35.132068 30758 init.cc:377] Logged in from keytab as kudu/127.24.183.193@KRBTEST.COM (short username kudu)
I20260809 15:07:35.133373 30758 webserver.cc:460] Webserver started at http://127.24.183.193:35295/ using document root <none> and password file <none>
I20260809 15:07:35.133998 30758 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:35.134099 30758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:35.134400 30758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:35.137439 30758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-0/data/instance:
uuid: "ef3519accbb44a7b9e703fed1c34a6c9"
format_stamp: "Formatted at 2026-08-09 15:07:35 on dist-test-slave-nj21"
server_key: "4e44c73fbe24c8f65ec70d27af11c8ea"
server_key_iv: "99aeed327c524e75c7844d66fc97c8c1"
server_key_version: "encryptionkey@0"
I20260809 15:07:35.138190 30758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance:
uuid: "ef3519accbb44a7b9e703fed1c34a6c9"
format_stamp: "Formatted at 2026-08-09 15:07:35 on dist-test-slave-nj21"
server_key: "4e44c73fbe24c8f65ec70d27af11c8ea"
server_key_iv: "99aeed327c524e75c7844d66fc97c8c1"
server_key_version: "encryptionkey@0"
I20260809 15:07:35.143167 30758 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 15:07:35.146714 30781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:35.148139 30758 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:07:35.148293 30758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-0/wal
uuid: "ef3519accbb44a7b9e703fed1c34a6c9"
format_stamp: "Formatted at 2026-08-09 15:07:35 on dist-test-slave-nj21"
server_key: "4e44c73fbe24c8f65ec70d27af11c8ea"
server_key_iv: "99aeed327c524e75c7844d66fc97c8c1"
server_key_version: "encryptionkey@0"
I20260809 15:07:35.148487 30758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:35.159664 30758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:35.163380 30758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:35.163638 30758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:35.164659 30758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:07:35.165966 30758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:07:35.166035 30758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:35.166153 30758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:07:35.166201 30758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:35.196580 30758 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.193:35155
I20260809 15:07:35.196614 30958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.193:35155 every 8 connection(s)
I20260809 15:07:35.197876 30758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
I20260809 15:07:35.202776 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 30758
I20260809 15:07:35.202979 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance
I20260809 15:07:35.203640 25311 external_mini_cluster.cc:1428] Setting key 646eed15940ee2dc74ed270d853be2c0
Aug 09 15:07:35 dist-test-slave-nj21 krb5kdc[30599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288055, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:35.217696 30962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:35.200325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:40089 (local address 127.24.183.254:41027)
0809 15:07:35.200771 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:35.200776 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:35.201644 (+ 868us) server_negotiation.cc:407] Connection header received
0809 15:07:35.203505 (+ 1861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:35.203511 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:35.203596 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:35.203763 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:35.205477 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:35.206486 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:35.208059 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:35.208599 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:35.208722 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:35.212066 (+ 3344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:35.212072 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:35.212076 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:35.212113 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:35.214723 (+ 2610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:35.215525 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:35.215528 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:35.215531 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:35.215643 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:35.216073 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:35.216076 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:35.216078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:35.216348 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:35.216458 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:35.217258 (+ 800us) server_negotiation.cc:299] Negotiation successful
0809 15:07:35.217435 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":216,"threads_started":1}
I20260809 15:07:35.219038 30963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:35.200816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41027 (local address 127.24.183.193:40089)
0809 15:07:35.201482 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:35.201529 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:35.203244 (+ 1715us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:35.203956 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:35.203964 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:35.204571 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:35.205301 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:35.205327 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:35.206655 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:35.206659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:35.207921 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:35.207938 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:35.208780 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:35.208784 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:35.208991 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:35.209614 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:35.209636 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:35.211858 (+ 2222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:35.214863 (+ 3005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:35.214871 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:35.214884 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:35.215435 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:35.215748 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:35.215753 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:35.215757 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:35.215985 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:35.216470 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:35.216479 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:35.216907 (+ 428us) client_negotiation.cc:769] Sending connection context
0809 15:07:35.217186 (+ 279us) client_negotiation.cc:240] Negotiation successful
0809 15:07:35.217688 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":248,"threads_started":1}
I20260809 15:07:35.220561 30960 heartbeater.cc:347] Connected to a master server at 127.24.183.254:41027
I20260809 15:07:35.220854 30960 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:35.221541 30960 heartbeater.cc:511] Master 127.24.183.254:41027 requested a full tablet report, sending...
I20260809 15:07:35.223909 30669 ts_manager.cc:194] Registered new tserver with Master: ef3519accbb44a7b9e703fed1c34a6c9 (127.24.183.193:35155)
I20260809 15:07:35.225988 30669 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.193@KRBTEST.COM'} at 127.24.183.193:40089
WARNING: no policy specified for kudu/127.24.183.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab.
Entry for principal kudu/127.24.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab.
I20260809 15:07:35.251838 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.194:0
--local_ip_for_outbound_sockets=127.24.183.194
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:41027
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:34107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:35.574848 30966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:35.575119 30966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:35.575228 30966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:35.582435 30966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:35.582537 30966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:35.582657 30966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.194
I20260809 15:07:35.595563 30966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:34107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:41027
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.194
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:35.597148 30966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:35.598749 30966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:35.613365 30974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:35.613410 30978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:35.613437 30975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:35.614259 30966 server_base.cc:1034] running on GCE node
I20260809 15:07:35.614794 30966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:35.615633 30966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:35.616860 30966 hybrid_clock.cc:648] HybridClock initialized: now 1786288055616809 us; error 35 us; skew 500 ppm
I20260809 15:07:35.617177 30966 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:35 dist-test-slave-nj21 krb5kdc[30599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288055, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:35.620273 30966 init.cc:377] Logged in from keytab as kudu/127.24.183.194@KRBTEST.COM (short username kudu)
I20260809 15:07:35.621681 30966 webserver.cc:460] Webserver started at http://127.24.183.194:46051/ using document root <none> and password file <none>
I20260809 15:07:35.622328 30966 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:35.622455 30966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:35.622748 30966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:35.625639 30966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-1/data/instance:
uuid: "7e75c9b7528248e9ba59beb32dbc0842"
format_stamp: "Formatted at 2026-08-09 15:07:35 on dist-test-slave-nj21"
server_key: "9041b97eeb601fa8b0a51c054155da16"
server_key_iv: "722abd759d2dabcdaa0cf85dca3fb0a3"
server_key_version: "encryptionkey@0"
I20260809 15:07:35.626398 30966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance:
uuid: "7e75c9b7528248e9ba59beb32dbc0842"
format_stamp: "Formatted at 2026-08-09 15:07:35 on dist-test-slave-nj21"
server_key: "9041b97eeb601fa8b0a51c054155da16"
server_key_iv: "722abd759d2dabcdaa0cf85dca3fb0a3"
server_key_version: "encryptionkey@0"
I20260809 15:07:35.631326 30966 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 15:07:35.634752 30989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:35.636195 30966 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:07:35.636329 30966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-1/wal
uuid: "7e75c9b7528248e9ba59beb32dbc0842"
format_stamp: "Formatted at 2026-08-09 15:07:35 on dist-test-slave-nj21"
server_key: "9041b97eeb601fa8b0a51c054155da16"
server_key_iv: "722abd759d2dabcdaa0cf85dca3fb0a3"
server_key_version: "encryptionkey@0"
I20260809 15:07:35.636518 30966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:35.665983 30966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:35.669821 30966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:35.670082 30966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:35.671006 30966 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:07:35.672444 30966 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:07:35.672518 30966 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:35.672607 30966 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:07:35.672657 30966 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:35.704790 30966 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.194:42217
I20260809 15:07:35.704830 31172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.194:42217 every 8 connection(s)
I20260809 15:07:35.706048 30966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
I20260809 15:07:35.709458 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 30966
I20260809 15:07:35.709662 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance
I20260809 15:07:35.710234 25311 external_mini_cluster.cc:1428] Setting key ba6b9354c14a35829a8f362f6b7ff03c
Aug 09 15:07:35 dist-test-slave-nj21 krb5kdc[30599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288055, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:35.725898 30962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:35.708544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:46385 (local address 127.24.183.254:41027)
0809 15:07:35.708725 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:35.708729 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:35.709977 (+ 1248us) server_negotiation.cc:407] Connection header received
0809 15:07:35.711811 (+ 1834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:35.711817 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:35.711900 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:35.711987 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:35.713426 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:35.714126 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:35.715638 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:35.716123 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:35.716237 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:35.719721 (+ 3484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:35.719728 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:35.719732 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:35.719772 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:35.722644 (+ 2872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:35.723528 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:35.723533 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:35.723535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:35.723726 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:35.724262 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:35.724266 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:35.724268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:35.724543 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:35.724662 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:35.725471 (+ 809us) server_negotiation.cc:299] Negotiation successful
0809 15:07:35.725642 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 15:07:35.727356 31177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:35.708990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41027 (local address 127.24.183.194:46385)
0809 15:07:35.709829 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:35.709875 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:35.711540 (+ 1665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:35.712219 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:35.712226 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:35.712706 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:35.713260 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:35.713277 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:35.714276 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:35.714280 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:35.715499 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:35.715511 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:35.716338 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:35.716346 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:35.716572 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:35.717179 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:35.717200 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:35.719438 (+ 2238us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:35.722830 (+ 3392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:35.722838 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:35.722853 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:35.723417 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:35.723848 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:35.723853 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:35.723858 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:35.724126 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:35.724698 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:35.724706 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:35.725106 (+ 400us) client_negotiation.cc:769] Sending connection context
0809 15:07:35.725403 (+ 297us) client_negotiation.cc:240] Negotiation successful
0809 15:07:35.725959 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":347,"threads_started":1}
I20260809 15:07:35.728966 31174 heartbeater.cc:347] Connected to a master server at 127.24.183.254:41027
I20260809 15:07:35.729250 31174 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:35.729984 31174 heartbeater.cc:511] Master 127.24.183.254:41027 requested a full tablet report, sending...
I20260809 15:07:35.731462 30669 ts_manager.cc:194] Registered new tserver with Master: 7e75c9b7528248e9ba59beb32dbc0842 (127.24.183.194:42217)
WARNING: no policy specified for kudu/127.24.183.195@KRBTEST.COM; defaulting to no policy
I20260809 15:07:35.732393 30669 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.194@KRBTEST.COM'} at 127.24.183.194:46385
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab.
Entry for principal kudu/127.24.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab.
I20260809 15:07:35.758865 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.195:0
--local_ip_for_outbound_sockets=127.24.183.195
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:41027
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:34107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:36.084112 31181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:36.084409 31181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:36.084558 31181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:36.091854 31181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:36.091969 31181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:36.092150 31181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.195
I20260809 15:07:36.101881 31181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:34107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:41027
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.195
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:36.103206 31181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:36.104557 31181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:36.117511 31197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:36.117550 31193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:36.117578 31194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:36.118245 31181 server_base.cc:1034] running on GCE node
I20260809 15:07:36.118848 31181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:36.119798 31181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:36.120993 31181 hybrid_clock.cc:648] HybridClock initialized: now 1786288056120954 us; error 34 us; skew 500 ppm
I20260809 15:07:36.121291 31181 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:36 dist-test-slave-nj21 krb5kdc[30599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288056, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:36.124249 31181 init.cc:377] Logged in from keytab as kudu/127.24.183.195@KRBTEST.COM (short username kudu)
I20260809 15:07:36.125628 31181 webserver.cc:460] Webserver started at http://127.24.183.195:35471/ using document root <none> and password file <none>
I20260809 15:07:36.126292 31181 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:36.126407 31181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:36.126705 31181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:36.129626 31181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-2/data/instance:
uuid: "bb3be57536d24a64adfe64bdba2f9018"
format_stamp: "Formatted at 2026-08-09 15:07:36 on dist-test-slave-nj21"
server_key: "1c93c7b9f8933edc6f598ef1e79c6e5d"
server_key_iv: "c10fa3321163c503161e450ce53f8cce"
server_key_version: "encryptionkey@0"
I20260809 15:07:36.130439 31181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance:
uuid: "bb3be57536d24a64adfe64bdba2f9018"
format_stamp: "Formatted at 2026-08-09 15:07:36 on dist-test-slave-nj21"
server_key: "1c93c7b9f8933edc6f598ef1e79c6e5d"
server_key_iv: "c10fa3321163c503161e450ce53f8cce"
server_key_version: "encryptionkey@0"
I20260809 15:07:36.135573 31181 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 15:07:36.139176 31211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:36.140594 31181 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:07:36.140764 31181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-2/wal
uuid: "bb3be57536d24a64adfe64bdba2f9018"
format_stamp: "Formatted at 2026-08-09 15:07:36 on dist-test-slave-nj21"
server_key: "1c93c7b9f8933edc6f598ef1e79c6e5d"
server_key_iv: "c10fa3321163c503161e450ce53f8cce"
server_key_version: "encryptionkey@0"
I20260809 15:07:36.140959 31181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:36.155249 31181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:36.159099 31181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:36.159375 31181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:36.160365 31181 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:07:36.161680 31181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:07:36.161758 31181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:36.161885 31181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:07:36.161948 31181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:36.193248 31181 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.195:35541
I20260809 15:07:36.193287 31380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.195:35541 every 8 connection(s)
I20260809 15:07:36.194532 31181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
I20260809 15:07:36.196555 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 31181
I20260809 15:07:36.196743 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance
I20260809 15:07:36.197177 25311 external_mini_cluster.cc:1428] Setting key 36b9ed93d2b914f64573a4dbcdb64477
Aug 09 15:07:36 dist-test-slave-nj21 krb5kdc[30599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288056, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:36.214383 30962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:36.198298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:57971 (local address 127.24.183.254:41027)
0809 15:07:36.198669 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:36.198674 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:36.198980 (+ 306us) server_negotiation.cc:407] Connection header received
0809 15:07:36.201319 (+ 2339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:36.201325 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:36.201407 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:36.201506 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:36.202850 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:36.203606 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:36.205074 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:36.205486 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:36.205575 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:36.208971 (+ 3396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:36.208975 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:36.208979 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:36.209010 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:36.211383 (+ 2373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:36.212216 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:36.212220 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:36.212223 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:36.212375 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:36.212863 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:36.212868 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:36.212871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:36.213230 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:36.213346 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:36.213952 (+ 606us) server_negotiation.cc:299] Negotiation successful
0809 15:07:36.214121 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245}
I20260809 15:07:36.215236 31384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:36.197741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41027 (local address 127.24.183.195:57971)
0809 15:07:36.198836 (+ 1095us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:36.198888 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:36.201078 (+ 2190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:36.201671 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:36.201678 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:36.202165 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:36.202705 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:36.202721 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:36.203756 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:36.203759 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:36.204962 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:36.204974 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:36.205645 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:36.205649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:36.205852 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:36.206464 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:36.206486 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:36.208783 (+ 2297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:36.211518 (+ 2735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:36.211527 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:36.211541 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:36.212110 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:36.212492 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:36.212497 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:36.212500 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:36.212744 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:36.213351 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:36.213357 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:36.213651 (+ 294us) client_negotiation.cc:769] Sending connection context
0809 15:07:36.213862 (+ 211us) client_negotiation.cc:240] Negotiation successful
0809 15:07:36.214224 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":795,"thread_start_us":376,"threads_started":1}
I20260809 15:07:36.216393 31381 heartbeater.cc:347] Connected to a master server at 127.24.183.254:41027
I20260809 15:07:36.216646 31381 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:36.217312 31381 heartbeater.cc:511] Master 127.24.183.254:41027 requested a full tablet report, sending...
I20260809 15:07:36.218652 30669 ts_manager.cc:194] Registered new tserver with Master: bb3be57536d24a64adfe64bdba2f9018 (127.24.183.195:35541)
I20260809 15:07:36.219432 30669 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.195@KRBTEST.COM'} at 127.24.183.195:57971
I20260809 15:07:36.225036 25311 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:07:36.228873 30960 heartbeater.cc:503] Master 127.24.183.254:41027 was elected leader, sending a full tablet report...
Aug 09 15:07:36 dist-test-slave-nj21 krb5kdc[30599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 15:07:36 dist-test-slave-nj21 krb5kdc[30599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.193@KRBTEST.COM
I20260809 15:07:36.276932 31392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:36.262615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41314 (local address 127.24.183.193:35155)
0809 15:07:36.263268 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:36.263276 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:36.263305 (+ 29us) server_negotiation.cc:407] Connection header received
0809 15:07:36.263383 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:36.263390 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:36.263609 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:36.263779 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:36.265107 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:36.266190 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:36.267379 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:36.268004 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:36.268096 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:36.271807 (+ 3711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:36.271819 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:36.271835 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:36.271874 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:36.274591 (+ 2717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:36.275140 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:36.275146 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:36.275149 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:36.275270 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:36.275553 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:36.275556 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:36.275558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:36.275948 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:36.276188 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:36.276439 (+ 251us) server_negotiation.cc:299] Negotiation successful
0809 15:07:36.276638 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":339,"threads_started":1}
Aug 09 15:07:36 dist-test-slave-nj21 krb5kdc[30599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:36.294982 30962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:36.281049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58668 (local address 127.24.183.254:41027)
0809 15:07:36.281296 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:36.281302 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:36.281495 (+ 193us) server_negotiation.cc:407] Connection header received
0809 15:07:36.281653 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:36.281657 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:36.281766 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:36.281857 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:36.283037 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:36.283857 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:36.285588 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:36.286247 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:36.286374 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:36.290049 (+ 3675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:36.290055 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:36.290058 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:36.290095 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:36.292495 (+ 2400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:36.293088 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:36.293094 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:36.293098 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:36.293230 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:36.293563 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:36.293568 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:36.293572 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:36.293897 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:36.294144 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:36.294456 (+ 312us) server_negotiation.cc:299] Negotiation successful
0809 15:07:36.294672 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260809 15:07:36.295569 30669 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58668
I20260809 15:07:36.296496 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 30758
I20260809 15:07:36.308579 31392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:36.297377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41330 (local address 127.24.183.193:35155)
0809 15:07:36.297571 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:36.297577 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:36.297783 (+ 206us) server_negotiation.cc:407] Connection header received
0809 15:07:36.297936 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:36.297942 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:36.298009 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:36.298091 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:36.299300 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:36.300040 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:36.301209 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:36.301640 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:36.301751 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:36.303890 (+ 2139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:36.303895 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:36.303899 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:36.303931 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:36.306235 (+ 2304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:36.306796 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:36.306800 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:36.306802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:36.306934 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:36.307272 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:36.307275 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:36.307277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:36.307515 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:36.307777 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:36.308072 (+ 295us) server_negotiation.cc:299] Negotiation successful
0809 15:07:36.308286 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 15:07:36.309122 30952 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:07:36.391611 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 30758
I20260809 15:07:36.414510 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 30966
Aug 09 15:07:36 dist-test-slave-nj21 krb5kdc[30599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.194@KRBTEST.COM
I20260809 15:07:36.429776 31399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:36.415351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42996 (local address 127.24.183.194:42217)
0809 15:07:36.415915 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:36.415922 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:36.415948 (+ 26us) server_negotiation.cc:407] Connection header received
0809 15:07:36.416012 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:36.416019 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:36.416232 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:36.416389 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:36.417616 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:36.418632 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:36.419963 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:36.420529 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:36.420646 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:36.424467 (+ 3821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:36.424478 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:36.424492 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:36.424524 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:36.427008 (+ 2484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:36.427650 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:36.427657 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:36.427659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:36.427819 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:36.428163 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:36.428168 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:36.428172 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:36.428521 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:36.428821 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:36.429126 (+ 305us) server_negotiation.cc:299] Negotiation successful
0809 15:07:36.429400 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":239,"threads_started":1}
I20260809 15:07:36.431059 31167 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:07:36.513103 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 30966
I20260809 15:07:36.538395 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 31181
Aug 09 15:07:36 dist-test-slave-nj21 krb5kdc[30599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.195@KRBTEST.COM
I20260809 15:07:36.554049 31401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:36.539396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40816 (local address 127.24.183.195:35541)
0809 15:07:36.540059 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:36.540066 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:36.540096 (+ 30us) server_negotiation.cc:407] Connection header received
0809 15:07:36.540156 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:36.540162 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:36.540382 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:36.540536 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:36.541839 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:36.542891 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:36.544403 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:36.545046 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:36.545160 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:36.549180 (+ 4020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:36.549191 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:36.549207 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:36.549240 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:36.551770 (+ 2530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:36.552345 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:36.552351 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:36.552353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:36.552479 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:36.552725 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:36.552727 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:36.552729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:36.553038 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:36.553272 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:36.553489 (+ 217us) server_negotiation.cc:299] Negotiation successful
0809 15:07:36.553689 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":352,"threads_started":1}
I20260809 15:07:36.555280 31374 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:07:36.638334 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 31181
I20260809 15:07:36.662662 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 30618
I20260809 15:07:36.664094 30714 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:07:36.770857 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 30618
2026-08-09T15:07:36Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4038 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:07:36 dist-test-slave-nj21 krb5kdc[31408](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:07:36 dist-test-slave-nj21 krb5kdc[31408](info): set up 2 sockets
Aug 09 15:07:36 dist-test-slave-nj21 krb5kdc[31408](info): commencing operation
krb5kdc: starting...
W20260809 15:07:37.976725 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.003s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:07:38 dist-test-slave-nj21 krb5kdc[31408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:07:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:07:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
I20260809 15:07:38.197031 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:36951
--webserver_interface=127.24.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:43099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:36951
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:38.530974 31428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:38.531248 31428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:38.531404 31428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:38.538650 31428 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:07:38.538753 31428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:38.538816 31428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:38.538899 31428 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:07:38.538968 31428 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:07:38.548864 31428 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:43099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:36951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:36951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:38.550159 31428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:38.551466 31428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:38.562686 31435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:38.562774 31436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:38.562666 31439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:38.563923 31428 server_base.cc:1034] running on GCE node
I20260809 15:07:38.564682 31428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:38.566078 31428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:38.567286 31428 hybrid_clock.cc:648] HybridClock initialized: now 1786288058567235 us; error 47 us; skew 500 ppm
I20260809 15:07:38.567637 31428 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:38 dist-test-slave-nj21 krb5kdc[31408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288058, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:38.570643 31428 init.cc:377] Logged in from keytab as kudu/127.24.183.254@KRBTEST.COM (short username kudu)
I20260809 15:07:38.571841 31428 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:38.571951 31428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:38.572268 31428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:38.575222 31428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "c2347b30bbcf4819b5657d75652eaa7a"
format_stamp: "Formatted at 2026-08-09 15:07:38 on dist-test-slave-nj21"
server_key: "249a197217fe2898e372aaae8df4153e"
server_key_iv: "7679edb98e5812ca5b0e244be048bd4e"
server_key_version: "encryptionkey@0"
I20260809 15:07:38.576084 31428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "c2347b30bbcf4819b5657d75652eaa7a"
format_stamp: "Formatted at 2026-08-09 15:07:38 on dist-test-slave-nj21"
server_key: "249a197217fe2898e372aaae8df4153e"
server_key_iv: "7679edb98e5812ca5b0e244be048bd4e"
server_key_version: "encryptionkey@0"
I20260809 15:07:38.581096 31428 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 15:07:38.584483 31448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:38.585741 31428 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:07:38.585903 31428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "c2347b30bbcf4819b5657d75652eaa7a"
format_stamp: "Formatted at 2026-08-09 15:07:38 on dist-test-slave-nj21"
server_key: "249a197217fe2898e372aaae8df4153e"
server_key_iv: "7679edb98e5812ca5b0e244be048bd4e"
server_key_version: "encryptionkey@0"
I20260809 15:07:38.586109 31428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:38.608762 31428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:38.613083 31428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:38.613346 31428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:38.631918 31428 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:36951
I20260809 15:07:38.631897 31528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:36951 every 8 connection(s)
I20260809 15:07:38.632812 31428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:07:38.636632 31529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:38.638023 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 31428
I20260809 15:07:38.638265 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:07:38.639110 25311 external_mini_cluster.cc:1428] Setting key 0eb033583dd402b2c9588084a7de3f14
I20260809 15:07:38.651648 31529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2347b30bbcf4819b5657d75652eaa7a: Bootstrap starting.
Aug 09 15:07:38 dist-test-slave-nj21 krb5kdc[31408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:38.655694 31529 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2347b30bbcf4819b5657d75652eaa7a: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:38.656836 31529 log.cc:824] T 00000000000000000000000000000000 P c2347b30bbcf4819b5657d75652eaa7a: Log is configured to *not* fsync() on all Append() calls
I20260809 15:07:38.660257 31529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2347b30bbcf4819b5657d75652eaa7a: No bootstrap required, opened a new log
I20260809 15:07:38.663282 31532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:38.641329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39194 (local address 127.24.183.254:36951)
0809 15:07:38.642214 (+ 885us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:38.642226 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:38.642271 (+ 45us) server_negotiation.cc:407] Connection header received
0809 15:07:38.643949 (+ 1678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:38.643973 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:38.644382 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:38.644706 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:38.646636 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:38.648032 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:38.649830 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:38.650499 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:38.650635 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:38.655468 (+ 4833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:38.655477 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:38.655492 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:38.655536 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:38.658948 (+ 3412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:38.659875 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:38.659883 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:38.659890 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:38.660071 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:38.660496 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:38.660500 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:38.660502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:38.661089 (+ 587us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:38.661304 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:38.661792 (+ 488us) server_negotiation.cc:299] Negotiation successful
0809 15:07:38.662150 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":397,"threads_started":1}
I20260809 15:07:38.668273 31529 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2347b30bbcf4819b5657d75652eaa7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2347b30bbcf4819b5657d75652eaa7a" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36951 } }
I20260809 15:07:38.668633 31529 raft_consensus.cc:374] T 00000000000000000000000000000000 P c2347b30bbcf4819b5657d75652eaa7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:38.668728 31529 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2347b30bbcf4819b5657d75652eaa7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2347b30bbcf4819b5657d75652eaa7a, State: Initialized, Role: FOLLOWER
I20260809 15:07:38.669313 31529 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2347b30bbcf4819b5657d75652eaa7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2347b30bbcf4819b5657d75652eaa7a" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36951 } }
I20260809 15:07:38.669561 31529 raft_consensus.cc:388] T 00000000000000000000000000000000 P c2347b30bbcf4819b5657d75652eaa7a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:07:38.669690 31529 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2347b30bbcf4819b5657d75652eaa7a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:07:38.669836 31529 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2347b30bbcf4819b5657d75652eaa7a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:38.672132 31529 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2347b30bbcf4819b5657d75652eaa7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2347b30bbcf4819b5657d75652eaa7a" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36951 } }
I20260809 15:07:38.672636 31529 leader_election.cc:302] T 00000000000000000000000000000000 P c2347b30bbcf4819b5657d75652eaa7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c2347b30bbcf4819b5657d75652eaa7a; no voters:
I20260809 15:07:38.673177 31529 leader_election.cc:288] T 00000000000000000000000000000000 P c2347b30bbcf4819b5657d75652eaa7a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:07:38.673307 31534 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2347b30bbcf4819b5657d75652eaa7a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:07:38.673990 31534 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2347b30bbcf4819b5657d75652eaa7a [term 1 LEADER]: Becoming Leader. State: Replica: c2347b30bbcf4819b5657d75652eaa7a, State: Running, Role: LEADER
I20260809 15:07:38.674531 31534 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2347b30bbcf4819b5657d75652eaa7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2347b30bbcf4819b5657d75652eaa7a" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36951 } }
I20260809 15:07:38.675257 31529 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2347b30bbcf4819b5657d75652eaa7a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:07:38.677323 31537 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2347b30bbcf4819b5657d75652eaa7a [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2347b30bbcf4819b5657d75652eaa7a. Latest consensus state: current_term: 1 leader_uuid: "c2347b30bbcf4819b5657d75652eaa7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2347b30bbcf4819b5657d75652eaa7a" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36951 } } }
I20260809 15:07:38.677824 31537 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2347b30bbcf4819b5657d75652eaa7a [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:38.677351 31536 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2347b30bbcf4819b5657d75652eaa7a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c2347b30bbcf4819b5657d75652eaa7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2347b30bbcf4819b5657d75652eaa7a" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36951 } } }
I20260809 15:07:38.678619 31536 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2347b30bbcf4819b5657d75652eaa7a [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:38.684243 31540 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:07:38.689718 31540 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:07:38.699074 31540 catalog_manager.cc:1347] Generated new cluster ID: da73be3f9efe478991f14413ed01c9e8
I20260809 15:07:38.699190 31540 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:07:38.711027 31540 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:07:38.712375 31540 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:07:38.722723 31540 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c2347b30bbcf4819b5657d75652eaa7a: Generated new TSK 0
I20260809 15:07:38.723567 31540 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab.
Entry for principal kudu/127.24.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab.
I20260809 15:07:38.783149 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.193:0
--local_ip_for_outbound_sockets=127.24.183.193
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:36951
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:43099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:39.120050 31562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:39.120358 31562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:39.120507 31562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:39.127833 31562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:39.127952 31562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:39.128110 31562 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.193
I20260809 15:07:39.138149 31562 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:43099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:36951
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.193
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:39.139551 31562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:39.140923 31562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:39.153559 31569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:39.153586 31573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:39.153630 31570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:39.154654 31562 server_base.cc:1034] running on GCE node
I20260809 15:07:39.155229 31562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:39.156234 31562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:39.157450 31562 hybrid_clock.cc:648] HybridClock initialized: now 1786288059157393 us; error 49 us; skew 500 ppm
I20260809 15:07:39.157766 31562 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:39 dist-test-slave-nj21 krb5kdc[31408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288059, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:39.160945 31562 init.cc:377] Logged in from keytab as kudu/127.24.183.193@KRBTEST.COM (short username kudu)
I20260809 15:07:39.162070 31562 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:39.162177 31562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:39.162578 31562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:39.165668 31562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-0/data/instance:
uuid: "b070fe6b41c6458695df0c6b5e072c50"
format_stamp: "Formatted at 2026-08-09 15:07:39 on dist-test-slave-nj21"
server_key: "74cb7eadde59f0ca9a2a57771a4b3268"
server_key_iv: "e74633e301d733ab95cba6f9b217768f"
server_key_version: "encryptionkey@0"
I20260809 15:07:39.166460 31562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance:
uuid: "b070fe6b41c6458695df0c6b5e072c50"
format_stamp: "Formatted at 2026-08-09 15:07:39 on dist-test-slave-nj21"
server_key: "74cb7eadde59f0ca9a2a57771a4b3268"
server_key_iv: "e74633e301d733ab95cba6f9b217768f"
server_key_version: "encryptionkey@0"
I20260809 15:07:39.171663 31562 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 15:07:39.175182 31584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:39.176668 31562 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:07:39.176831 31562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-0/wal
uuid: "b070fe6b41c6458695df0c6b5e072c50"
format_stamp: "Formatted at 2026-08-09 15:07:39 on dist-test-slave-nj21"
server_key: "74cb7eadde59f0ca9a2a57771a4b3268"
server_key_iv: "e74633e301d733ab95cba6f9b217768f"
server_key_version: "encryptionkey@0"
I20260809 15:07:39.177048 31562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:39.198655 31562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:39.202571 31562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:39.202826 31562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:39.203791 31562 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:07:39.205142 31562 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:07:39.205238 31562 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:39.205355 31562 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:07:39.205428 31562 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:39.238273 31562 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.193:37311
I20260809 15:07:39.238322 31750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.193:37311 every 8 connection(s)
I20260809 15:07:39.239163 31562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
I20260809 15:07:39.245828 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 31562
I20260809 15:07:39.246066 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance
I20260809 15:07:39.246886 25311 external_mini_cluster.cc:1428] Setting key 5ee15487f473dae0b0007d5d30611842
Aug 09 15:07:39 dist-test-slave-nj21 krb5kdc[31408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288059, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:39.263283 31756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:39.241574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:53923 (local address 127.24.183.254:36951)
0809 15:07:39.242003 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:39.242008 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:39.242962 (+ 954us) server_negotiation.cc:407] Connection header received
0809 15:07:39.244832 (+ 1870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:39.244838 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:39.244916 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:39.245016 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:39.246991 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:39.247800 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:39.249891 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:39.250306 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:39.250402 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:39.257370 (+ 6968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:39.257375 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:39.257379 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:39.257415 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:39.260314 (+ 2899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:39.261028 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:39.261033 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:39.261036 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:39.261180 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:39.261542 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:39.261546 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:39.261550 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:39.261899 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:39.262040 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:39.262698 (+ 658us) server_negotiation.cc:299] Negotiation successful
0809 15:07:39.262936 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":207,"threads_started":1}
I20260809 15:07:39.264119 31757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:39.242107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:36951 (local address 127.24.183.193:53923)
0809 15:07:39.242795 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:39.242840 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:39.244558 (+ 1718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:39.245240 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:39.245248 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:39.245978 (+ 730us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:39.246782 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:39.246807 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:39.247956 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:39.247962 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:39.249727 (+ 1765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:39.249742 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:39.250534 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:39.250543 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:39.250813 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:39.254383 (+ 3570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:39.254415 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:39.257109 (+ 2694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:39.260457 (+ 3348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:39.260464 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:39.260478 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:39.260909 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:39.261268 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:39.261271 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:39.261273 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:39.261445 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:39.261999 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:39.262005 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:39.262317 (+ 312us) client_negotiation.cc:769] Sending connection context
0809 15:07:39.262542 (+ 225us) client_negotiation.cc:240] Negotiation successful
0809 15:07:39.262947 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":283,"threads_started":1}
I20260809 15:07:39.265601 31753 heartbeater.cc:347] Connected to a master server at 127.24.183.254:36951
I20260809 15:07:39.265949 31753 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:39.266739 31753 heartbeater.cc:511] Master 127.24.183.254:36951 requested a full tablet report, sending...
I20260809 15:07:39.269045 31474 ts_manager.cc:194] Registered new tserver with Master: b070fe6b41c6458695df0c6b5e072c50 (127.24.183.193:37311)
WARNING: no policy specified for kudu/127.24.183.194@KRBTEST.COM; defaulting to no policy
I20260809 15:07:39.270494 31474 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.193@KRBTEST.COM'} at 127.24.183.193:53923
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab.
Entry for principal kudu/127.24.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab.
I20260809 15:07:39.296854 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.194:0
--local_ip_for_outbound_sockets=127.24.183.194
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:36951
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:43099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:39.625373 31766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:39.625649 31766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:39.625802 31766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:39.632942 31766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:39.633060 31766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:39.633219 31766 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.194
I20260809 15:07:39.642597 31766 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:43099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:36951
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.194
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:39.643821 31766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:39.645080 31766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:39.657171 31778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:39.657143 31782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:39.657107 31779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:39.657927 31766 server_base.cc:1034] running on GCE node
I20260809 15:07:39.658488 31766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:39.659382 31766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:39.660576 31766 hybrid_clock.cc:648] HybridClock initialized: now 1786288059660522 us; error 43 us; skew 500 ppm
I20260809 15:07:39.660877 31766 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:39 dist-test-slave-nj21 krb5kdc[31408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288059, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:39.663868 31766 init.cc:377] Logged in from keytab as kudu/127.24.183.194@KRBTEST.COM (short username kudu)
I20260809 15:07:39.664947 31766 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:39.665037 31766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:39.665289 31766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:39.668119 31766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-1/data/instance:
uuid: "5192b47149514d4483248c0a10750c00"
format_stamp: "Formatted at 2026-08-09 15:07:39 on dist-test-slave-nj21"
server_key: "2455f2b0358b0403e059fe8b70dbe5d9"
server_key_iv: "f172c957435cb6baff7cfb9ae08a1d57"
server_key_version: "encryptionkey@0"
I20260809 15:07:39.668851 31766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance:
uuid: "5192b47149514d4483248c0a10750c00"
format_stamp: "Formatted at 2026-08-09 15:07:39 on dist-test-slave-nj21"
server_key: "2455f2b0358b0403e059fe8b70dbe5d9"
server_key_iv: "f172c957435cb6baff7cfb9ae08a1d57"
server_key_version: "encryptionkey@0"
I20260809 15:07:39.673668 31766 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 15:07:39.683326 31791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:39.685141 31766 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.002s sys 0.000s
I20260809 15:07:39.685322 31766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-1/wal
uuid: "5192b47149514d4483248c0a10750c00"
format_stamp: "Formatted at 2026-08-09 15:07:39 on dist-test-slave-nj21"
server_key: "2455f2b0358b0403e059fe8b70dbe5d9"
server_key_iv: "f172c957435cb6baff7cfb9ae08a1d57"
server_key_version: "encryptionkey@0"
I20260809 15:07:39.685577 31766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:39.702800 31766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:39.706408 31766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:39.706619 31766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:39.707479 31766 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:07:39.708792 31766 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:07:39.708881 31766 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:39.708964 31766 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:07:39.709017 31766 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:39.737578 31766 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.194:33341
I20260809 15:07:39.737632 31968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.194:33341 every 8 connection(s)
I20260809 15:07:39.738377 31766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
I20260809 15:07:39.745010 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 31766
I20260809 15:07:39.745194 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance
I20260809 15:07:39.745723 25311 external_mini_cluster.cc:1428] Setting key 0e7fd89a1fa12e29ca73d4a15af1cff3
Aug 09 15:07:39 dist-test-slave-nj21 krb5kdc[31408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288059, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:39.757812 31756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:39.740707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:38819 (local address 127.24.183.254:36951)
0809 15:07:39.740926 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:39.740933 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:39.742049 (+ 1116us) server_negotiation.cc:407] Connection header received
0809 15:07:39.743869 (+ 1820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:39.743874 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:39.743948 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:39.744028 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:39.745492 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:39.746445 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:39.747982 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:39.748623 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:39.748767 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:39.752306 (+ 3539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:39.752311 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:39.752314 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:39.752349 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:39.754726 (+ 2377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:39.755567 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:39.755570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:39.755573 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:39.755735 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:39.756137 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:39.756141 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:39.756144 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:39.756493 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:39.756644 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:39.757248 (+ 604us) server_negotiation.cc:299] Negotiation successful
0809 15:07:39.757475 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 15:07:39.758570 31971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:39.741194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:36951 (local address 127.24.183.194:38819)
0809 15:07:39.741898 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:39.741941 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:39.743564 (+ 1623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:39.744207 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:39.744214 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:39.744707 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:39.745309 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:39.745333 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:39.746585 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:39.746589 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:39.747836 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:39.747848 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:39.748876 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:39.748883 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:39.749127 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:39.749757 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:39.749779 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:39.752084 (+ 2305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:39.754886 (+ 2802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:39.754895 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:39.754911 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:39.755464 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:39.755834 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:39.755839 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:39.755843 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:39.756030 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:39.756599 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:39.756604 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:39.756902 (+ 298us) client_negotiation.cc:769] Sending connection context
0809 15:07:39.757111 (+ 209us) client_negotiation.cc:240] Negotiation successful
0809 15:07:39.757487 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":289,"threads_started":1}
I20260809 15:07:39.760046 31969 heartbeater.cc:347] Connected to a master server at 127.24.183.254:36951
I20260809 15:07:39.760381 31969 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:39.761200 31969 heartbeater.cc:511] Master 127.24.183.254:36951 requested a full tablet report, sending...
I20260809 15:07:39.762676 31474 ts_manager.cc:194] Registered new tserver with Master: 5192b47149514d4483248c0a10750c00 (127.24.183.194:33341)
I20260809 15:07:39.763479 31474 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.194@KRBTEST.COM'} at 127.24.183.194:38819
WARNING: no policy specified for kudu/127.24.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab.
Entry for principal kudu/127.24.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab.
I20260809 15:07:39.794416 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.195:0
--local_ip_for_outbound_sockets=127.24.183.195
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:36951
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:43099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:40.117962 31975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:40.118222 31975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:40.118353 31975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:40.125393 31975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:40.125505 31975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:40.125660 31975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.195
I20260809 15:07:40.135125 31975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:43099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:36951
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.195
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:40.136500 31975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:40.137801 31975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:40.150359 31983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:40.150436 31985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:40.150811 31990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:40.151232 31975 server_base.cc:1034] running on GCE node
I20260809 15:07:40.151918 31975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:40.152879 31975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:40.154078 31975 hybrid_clock.cc:648] HybridClock initialized: now 1786288060154026 us; error 45 us; skew 500 ppm
I20260809 15:07:40.154376 31975 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:40 dist-test-slave-nj21 krb5kdc[31408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288060, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:40.157433 31975 init.cc:377] Logged in from keytab as kudu/127.24.183.195@KRBTEST.COM (short username kudu)
I20260809 15:07:40.158525 31975 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:40.158628 31975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:40.158948 31975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:40.162025 31975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-2/data/instance:
uuid: "66807110b9b54b5b9e7e1ad75b93dcf0"
format_stamp: "Formatted at 2026-08-09 15:07:40 on dist-test-slave-nj21"
server_key: "e4e6971586f9bbfb2727176c17fd0764"
server_key_iv: "32ba01fbad2d9c45f543aaeddcad839f"
server_key_version: "encryptionkey@0"
I20260809 15:07:40.162797 31975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance:
uuid: "66807110b9b54b5b9e7e1ad75b93dcf0"
format_stamp: "Formatted at 2026-08-09 15:07:40 on dist-test-slave-nj21"
server_key: "e4e6971586f9bbfb2727176c17fd0764"
server_key_iv: "32ba01fbad2d9c45f543aaeddcad839f"
server_key_version: "encryptionkey@0"
I20260809 15:07:40.167546 31975 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260809 15:07:40.170874 32000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:40.172137 31975 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:07:40.172286 31975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-2/wal
uuid: "66807110b9b54b5b9e7e1ad75b93dcf0"
format_stamp: "Formatted at 2026-08-09 15:07:40 on dist-test-slave-nj21"
server_key: "e4e6971586f9bbfb2727176c17fd0764"
server_key_iv: "32ba01fbad2d9c45f543aaeddcad839f"
server_key_version: "encryptionkey@0"
I20260809 15:07:40.172487 31975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:40.201869 31975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:40.205688 31975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:40.205963 31975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:40.206893 31975 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:07:40.208323 31975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:07:40.208424 31975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:40.208532 31975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:07:40.208611 31975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:40.239642 31975 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.195:45377
I20260809 15:07:40.239715 32179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.195:45377 every 8 connection(s)
I20260809 15:07:40.240501 31975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
I20260809 15:07:40.241837 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 31975
I20260809 15:07:40.242019 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance
I20260809 15:07:40.242420 25311 external_mini_cluster.cc:1428] Setting key ceccbd3facd391d10d0d3d463dd72d4e
Aug 09 15:07:40 dist-test-slave-nj21 krb5kdc[31408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288060, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:40.260991 31756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:40.242959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:47391 (local address 127.24.183.254:36951)
0809 15:07:40.243168 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:40.243175 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:40.245447 (+ 2272us) server_negotiation.cc:407] Connection header received
0809 15:07:40.247382 (+ 1935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:40.247388 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:40.247503 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:40.247589 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:40.249073 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:40.249819 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:40.251332 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:40.251873 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:40.251988 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:40.255295 (+ 3307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:40.255300 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:40.255304 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:40.255339 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:40.257994 (+ 2655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:40.258660 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:40.258665 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:40.258669 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:40.258799 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:40.259203 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:40.259208 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:40.259211 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:40.259589 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:40.259758 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:40.260428 (+ 670us) server_negotiation.cc:299] Negotiation successful
0809 15:07:40.260648 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 15:07:40.261937 32185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:40.243413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:36951 (local address 127.24.183.195:47391)
0809 15:07:40.245304 (+ 1891us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:40.245341 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:40.247125 (+ 1784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:40.247863 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:40.247870 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:40.248356 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:40.248893 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:40.248910 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:40.249960 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:40.249966 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:40.251187 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:40.251200 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:40.252100 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:40.252109 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:40.252328 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:40.252957 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:40.252985 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:40.255083 (+ 2098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:40.258117 (+ 3034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:40.258126 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:40.258141 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:40.258547 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:40.258903 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:40.258908 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:40.258912 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:40.259096 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:40.259761 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:40.259769 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:40.260081 (+ 312us) client_negotiation.cc:769] Sending connection context
0809 15:07:40.260330 (+ 249us) client_negotiation.cc:240] Negotiation successful
0809 15:07:40.260834 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1663,"thread_start_us":1475,"threads_started":1}
I20260809 15:07:40.263218 32181 heartbeater.cc:347] Connected to a master server at 127.24.183.254:36951
I20260809 15:07:40.263509 32181 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:40.264196 32181 heartbeater.cc:511] Master 127.24.183.254:36951 requested a full tablet report, sending...
I20260809 15:07:40.265555 31474 ts_manager.cc:194] Registered new tserver with Master: 66807110b9b54b5b9e7e1ad75b93dcf0 (127.24.183.195:45377)
I20260809 15:07:40.266598 31474 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.195@KRBTEST.COM'} at 127.24.183.195:47391
I20260809 15:07:40.270246 25311 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:07:40.273442 31753 heartbeater.cc:503] Master 127.24.183.254:36951 was elected leader, sending a full tablet report...
I20260809 15:07:40.287139 31756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:40.275009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39204 (local address 127.24.183.254:36951)
0809 15:07:40.275165 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:40.275169 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:40.275713 (+ 544us) server_negotiation.cc:407] Connection header received
0809 15:07:40.275881 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:40.275887 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:40.275960 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:40.276044 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:40.277462 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:40.278154 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:40.279433 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:40.279931 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:40.280082 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:40.282266 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:40.282271 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:40.282274 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:40.282304 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:40.284705 (+ 2401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:40.285316 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:40.285322 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:40.285325 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:40.285460 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:40.285789 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:40.285794 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:40.285796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:40.286096 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:40.286324 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:40.286649 (+ 325us) server_negotiation.cc:299] Negotiation successful
0809 15:07:40.286855 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 15:07:40.290993 31474 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39204:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:07:40.298851 31474 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:07:40.322041 32201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:40.310836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:33341 (local address 127.0.0.1:54134)
0809 15:07:40.311859 (+ 1023us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:40.311907 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:40.312109 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:40.314061 (+ 1952us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:40.314069 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:40.314108 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:07:40.314559 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:40.314574 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:40.316804 (+ 2230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:40.316810 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:40.319189 (+ 2379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:40.319204 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:40.321059 (+ 1855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:40.321065 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:40.321252 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:40.321302 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 15:07:40.321430 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 15:07:40.321605 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"mutex_wait_us":49,"thread_start_us":518,"threads_started":1}
I20260809 15:07:40.322057 32204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:40.311144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54134 (local address 127.24.183.194:33341)
0809 15:07:40.313369 (+ 2225us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:40.313380 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:40.313412 (+ 32us) server_negotiation.cc:407] Connection header received
0809 15:07:40.313531 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:40.313540 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:40.313810 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:40.314771 (+ 961us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:07:40.315017 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:40.316591 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:40.319409 (+ 2818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:40.320840 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:40.321024 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:40.321210 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:40.321506 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 15:07:40.321682 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2082,"thread_start_us":1921,"threads_started":1}
I20260809 15:07:40.323334 32209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:40.312086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51568 (local address 127.24.183.193:37311)
0809 15:07:40.312917 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:40.312928 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:40.313839 (+ 911us) server_negotiation.cc:407] Connection header received
0809 15:07:40.314151 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:40.314161 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:40.314398 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:40.314580 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:07:40.315701 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:40.317255 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:40.319879 (+ 2624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:40.321351 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:40.321556 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:40.321717 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:40.322843 (+ 1126us) server_negotiation.cc:299] Negotiation successful
0809 15:07:40.322999 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":440,"threads_started":1}
I20260809 15:07:40.323398 32206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:40.312135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:45377 (local address 127.0.0.1:40082)
0809 15:07:40.315718 (+ 3583us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:40.315734 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:40.315849 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:40.316544 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:40.316549 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:40.316572 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:07:40.317007 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:40.317016 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:40.318905 (+ 1889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:40.318910 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:40.320537 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:40.320549 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:40.322459 (+ 1910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:40.322465 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:40.322684 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:40.322717 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 15:07:40.322796 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 15:07:40.322921 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3428,"mutex_wait_us":27,"thread_start_us":3287,"threads_started":1}
I20260809 15:07:40.323441 32205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:40.311457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:37311 (local address 127.0.0.1:51568)
0809 15:07:40.313708 (+ 2251us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:40.313735 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:40.313954 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:40.314878 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:40.314882 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:40.314908 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:07:40.315426 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:40.315441 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:40.317399 (+ 1958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:40.317405 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:40.319661 (+ 2256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:40.319709 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:40.322422 (+ 2713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:40.322436 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:40.322623 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:40.322687 (+ 64us) client_negotiation.cc:769] Sending connection context
0809 15:07:40.322785 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 15:07:40.322914 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2113,"mutex_wait_us":27,"thread_start_us":1968,"threads_started":1}
I20260809 15:07:40.324090 32210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:40.312738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40082 (local address 127.24.183.195:45377)
0809 15:07:40.314064 (+ 1326us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:40.314072 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:40.315986 (+ 1914us) server_negotiation.cc:407] Connection header received
0809 15:07:40.316097 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:40.316104 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:40.316361 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:40.316539 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:07:40.317194 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:40.318720 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:40.320679 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:40.321974 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:40.322153 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:40.322309 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:40.323600 (+ 1291us) server_negotiation.cc:299] Negotiation successful
0809 15:07:40.323780 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1164,"thread_start_us":999,"threads_started":1}
I20260809 15:07:40.325958 31876 tablet_service.cc:1467] Processing CreateTablet for tablet 4110e82df90741f9a4078929d72ad9e3 (DEFAULT_TABLE table=test-table [id=04586c18400f4b90af5704416f42c276]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:07:40.327170 31876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4110e82df90741f9a4078929d72ad9e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:40.330039 32078 tablet_service.cc:1467] Processing CreateTablet for tablet 4110e82df90741f9a4078929d72ad9e3 (DEFAULT_TABLE table=test-table [id=04586c18400f4b90af5704416f42c276]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:07:40.330119 31661 tablet_service.cc:1467] Processing CreateTablet for tablet 4110e82df90741f9a4078929d72ad9e3 (DEFAULT_TABLE table=test-table [id=04586c18400f4b90af5704416f42c276]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:07:40.331352 32078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4110e82df90741f9a4078929d72ad9e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:40.331718 31661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4110e82df90741f9a4078929d72ad9e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:40.342650 32214 tablet_bootstrap.cc:492] T 4110e82df90741f9a4078929d72ad9e3 P 66807110b9b54b5b9e7e1ad75b93dcf0: Bootstrap starting.
I20260809 15:07:40.343155 32211 tablet_bootstrap.cc:492] T 4110e82df90741f9a4078929d72ad9e3 P 5192b47149514d4483248c0a10750c00: Bootstrap starting.
I20260809 15:07:40.346619 32211 tablet_bootstrap.cc:654] T 4110e82df90741f9a4078929d72ad9e3 P 5192b47149514d4483248c0a10750c00: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:40.346923 32214 tablet_bootstrap.cc:654] T 4110e82df90741f9a4078929d72ad9e3 P 66807110b9b54b5b9e7e1ad75b93dcf0: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:40.347504 32215 tablet_bootstrap.cc:492] T 4110e82df90741f9a4078929d72ad9e3 P b070fe6b41c6458695df0c6b5e072c50: Bootstrap starting.
I20260809 15:07:40.348022 32211 log.cc:824] T 4110e82df90741f9a4078929d72ad9e3 P 5192b47149514d4483248c0a10750c00: Log is configured to *not* fsync() on all Append() calls
I20260809 15:07:40.348280 32214 log.cc:824] T 4110e82df90741f9a4078929d72ad9e3 P 66807110b9b54b5b9e7e1ad75b93dcf0: Log is configured to *not* fsync() on all Append() calls
I20260809 15:07:40.351577 32215 tablet_bootstrap.cc:654] T 4110e82df90741f9a4078929d72ad9e3 P b070fe6b41c6458695df0c6b5e072c50: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:40.351611 32211 tablet_bootstrap.cc:492] T 4110e82df90741f9a4078929d72ad9e3 P 5192b47149514d4483248c0a10750c00: No bootstrap required, opened a new log
I20260809 15:07:40.351910 32211 ts_tablet_manager.cc:1397] T 4110e82df90741f9a4078929d72ad9e3 P 5192b47149514d4483248c0a10750c00: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260809 15:07:40.352509 32214 tablet_bootstrap.cc:492] T 4110e82df90741f9a4078929d72ad9e3 P 66807110b9b54b5b9e7e1ad75b93dcf0: No bootstrap required, opened a new log
I20260809 15:07:40.352665 32215 log.cc:824] T 4110e82df90741f9a4078929d72ad9e3 P b070fe6b41c6458695df0c6b5e072c50: Log is configured to *not* fsync() on all Append() calls
I20260809 15:07:40.352761 32214 ts_tablet_manager.cc:1397] T 4110e82df90741f9a4078929d72ad9e3 P 66807110b9b54b5b9e7e1ad75b93dcf0: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.001s
I20260809 15:07:40.355093 32215 tablet_bootstrap.cc:492] T 4110e82df90741f9a4078929d72ad9e3 P b070fe6b41c6458695df0c6b5e072c50: No bootstrap required, opened a new log
I20260809 15:07:40.355350 32215 ts_tablet_manager.cc:1397] T 4110e82df90741f9a4078929d72ad9e3 P b070fe6b41c6458695df0c6b5e072c50: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260809 15:07:40.362123 32211 raft_consensus.cc:348] T 4110e82df90741f9a4078929d72ad9e3 P 5192b47149514d4483248c0a10750c00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5192b47149514d4483248c0a10750c00" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 33341 } } peers { permanent_uuid: "66807110b9b54b5b9e7e1ad75b93dcf0" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45377 } } peers { permanent_uuid: "b070fe6b41c6458695df0c6b5e072c50" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 37311 } }
I20260809 15:07:40.362546 32211 raft_consensus.cc:374] T 4110e82df90741f9a4078929d72ad9e3 P 5192b47149514d4483248c0a10750c00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:40.362656 32211 raft_consensus.cc:729] T 4110e82df90741f9a4078929d72ad9e3 P 5192b47149514d4483248c0a10750c00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5192b47149514d4483248c0a10750c00, State: Initialized, Role: FOLLOWER
I20260809 15:07:40.362797 32214 raft_consensus.cc:348] T 4110e82df90741f9a4078929d72ad9e3 P 66807110b9b54b5b9e7e1ad75b93dcf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5192b47149514d4483248c0a10750c00" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 33341 } } peers { permanent_uuid: "66807110b9b54b5b9e7e1ad75b93dcf0" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45377 } } peers { permanent_uuid: "b070fe6b41c6458695df0c6b5e072c50" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 37311 } }
I20260809 15:07:40.363197 32214 raft_consensus.cc:374] T 4110e82df90741f9a4078929d72ad9e3 P 66807110b9b54b5b9e7e1ad75b93dcf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:40.363013 32215 raft_consensus.cc:348] T 4110e82df90741f9a4078929d72ad9e3 P b070fe6b41c6458695df0c6b5e072c50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5192b47149514d4483248c0a10750c00" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 33341 } } peers { permanent_uuid: "66807110b9b54b5b9e7e1ad75b93dcf0" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45377 } } peers { permanent_uuid: "b070fe6b41c6458695df0c6b5e072c50" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 37311 } }
I20260809 15:07:40.363301 32214 raft_consensus.cc:729] T 4110e82df90741f9a4078929d72ad9e3 P 66807110b9b54b5b9e7e1ad75b93dcf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66807110b9b54b5b9e7e1ad75b93dcf0, State: Initialized, Role: FOLLOWER
I20260809 15:07:40.363257 32211 consensus_queue.cc:260] T 4110e82df90741f9a4078929d72ad9e3 P 5192b47149514d4483248c0a10750c00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5192b47149514d4483248c0a10750c00" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 33341 } } peers { permanent_uuid: "66807110b9b54b5b9e7e1ad75b93dcf0" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45377 } } peers { permanent_uuid: "b070fe6b41c6458695df0c6b5e072c50" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 37311 } }
I20260809 15:07:40.363462 32215 raft_consensus.cc:374] T 4110e82df90741f9a4078929d72ad9e3 P b070fe6b41c6458695df0c6b5e072c50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:40.363729 32215 raft_consensus.cc:729] T 4110e82df90741f9a4078929d72ad9e3 P b070fe6b41c6458695df0c6b5e072c50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b070fe6b41c6458695df0c6b5e072c50, State: Initialized, Role: FOLLOWER
I20260809 15:07:40.363942 32214 consensus_queue.cc:260] T 4110e82df90741f9a4078929d72ad9e3 P 66807110b9b54b5b9e7e1ad75b93dcf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5192b47149514d4483248c0a10750c00" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 33341 } } peers { permanent_uuid: "66807110b9b54b5b9e7e1ad75b93dcf0" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45377 } } peers { permanent_uuid: "b070fe6b41c6458695df0c6b5e072c50" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 37311 } }
I20260809 15:07:40.364446 31969 heartbeater.cc:503] Master 127.24.183.254:36951 was elected leader, sending a full tablet report...
I20260809 15:07:40.364549 32215 consensus_queue.cc:260] T 4110e82df90741f9a4078929d72ad9e3 P b070fe6b41c6458695df0c6b5e072c50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5192b47149514d4483248c0a10750c00" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 33341 } } peers { permanent_uuid: "66807110b9b54b5b9e7e1ad75b93dcf0" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45377 } } peers { permanent_uuid: "b070fe6b41c6458695df0c6b5e072c50" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 37311 } }
I20260809 15:07:40.365049 32211 ts_tablet_manager.cc:1428] T 4110e82df90741f9a4078929d72ad9e3 P 5192b47149514d4483248c0a10750c00: Time spent starting tablet: real 0.013s user 0.009s sys 0.003s
I20260809 15:07:40.367038 32214 ts_tablet_manager.cc:1428] T 4110e82df90741f9a4078929d72ad9e3 P 66807110b9b54b5b9e7e1ad75b93dcf0: Time spent starting tablet: real 0.014s user 0.012s sys 0.002s
I20260809 15:07:40.367488 32181 heartbeater.cc:503] Master 127.24.183.254:36951 was elected leader, sending a full tablet report...
I20260809 15:07:40.368883 32215 ts_tablet_manager.cc:1428] T 4110e82df90741f9a4078929d72ad9e3 P b070fe6b41c6458695df0c6b5e072c50: Time spent starting tablet: real 0.013s user 0.010s sys 0.005s
I20260809 15:07:40.399330 32223 raft_consensus.cc:482] T 4110e82df90741f9a4078929d72ad9e3 P 66807110b9b54b5b9e7e1ad75b93dcf0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:07:40.399610 32223 raft_consensus.cc:504] T 4110e82df90741f9a4078929d72ad9e3 P 66807110b9b54b5b9e7e1ad75b93dcf0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5192b47149514d4483248c0a10750c00" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 33341 } } peers { permanent_uuid: "66807110b9b54b5b9e7e1ad75b93dcf0" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45377 } } peers { permanent_uuid: "b070fe6b41c6458695df0c6b5e072c50" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 37311 } }
I20260809 15:07:40.401270 32223 leader_election.cc:288] T 4110e82df90741f9a4078929d72ad9e3 P 66807110b9b54b5b9e7e1ad75b93dcf0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5192b47149514d4483248c0a10750c00 (127.24.183.194:33341), b070fe6b41c6458695df0c6b5e072c50 (127.24.183.193:37311)
I20260809 15:07:40.406790 32204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:40.401783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:42827 (local address 127.24.183.194:33341)
0809 15:07:40.402136 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:40.402143 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:40.402164 (+ 21us) server_negotiation.cc:407] Connection header received
0809 15:07:40.402238 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:40.402244 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:40.402321 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:40.402410 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:07:40.402827 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:40.403537 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:40.405153 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:40.405935 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:40.406072 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:40.406146 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:40.406419 (+ 273us) server_negotiation.cc:299] Negotiation successful
0809 15:07:40.406538 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218}
I20260809 15:07:40.406795 32185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:40.401471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:33341 (local address 127.24.183.195:42827)
0809 15:07:40.401678 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:40.401698 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:40.401812 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:40.402450 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:40.402456 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:40.402484 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:07:40.402659 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:40.402668 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:40.403691 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:40.403697 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:40.404993 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:40.405005 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:40.406084 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:40.406089 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:40.406269 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:40.406294 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 15:07:40.406374 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 15:07:40.406506 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 15:07:40.407294 32209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:40.402026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:41543 (local address 127.24.183.193:37311)
0809 15:07:40.402240 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:40.402246 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:40.402425 (+ 179us) server_negotiation.cc:407] Connection header received
0809 15:07:40.402584 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:40.402588 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:40.402645 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:40.402719 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:07:40.403245 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:40.404004 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:40.405686 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:40.406441 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:40.406581 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:40.406652 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:40.406968 (+ 316us) server_negotiation.cc:299] Negotiation successful
0809 15:07:40.407090 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 15:07:40.407871 32226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:40.401632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:37311 (local address 127.24.183.195:41543)
0809 15:07:40.402332 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:40.402345 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:40.402459 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:40.402746 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:40.402751 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:40.402774 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:07:40.403083 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:40.403092 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:40.404143 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:40.404148 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:40.405535 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:40.405547 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:40.406632 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:40.406638 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:40.406777 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:40.406804 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 15:07:40.407469 (+ 665us) client_negotiation.cc:240] Negotiation successful
0809 15:07:40.407577 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":419,"threads_started":1}
I20260809 15:07:40.407822 31901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4110e82df90741f9a4078929d72ad9e3" candidate_uuid: "66807110b9b54b5b9e7e1ad75b93dcf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5192b47149514d4483248c0a10750c00" is_pre_election: true
I20260809 15:07:40.408340 31901 raft_consensus.cc:2454] T 4110e82df90741f9a4078929d72ad9e3 P 5192b47149514d4483248c0a10750c00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66807110b9b54b5b9e7e1ad75b93dcf0 in term 0.
I20260809 15:07:40.408806 31690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4110e82df90741f9a4078929d72ad9e3" candidate_uuid: "66807110b9b54b5b9e7e1ad75b93dcf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b070fe6b41c6458695df0c6b5e072c50" is_pre_election: true
I20260809 15:07:40.408999 32006 leader_election.cc:302] T 4110e82df90741f9a4078929d72ad9e3 P 66807110b9b54b5b9e7e1ad75b93dcf0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5192b47149514d4483248c0a10750c00, 66807110b9b54b5b9e7e1ad75b93dcf0; no voters:
I20260809 15:07:40.409380 32223 raft_consensus.cc:2781] T 4110e82df90741f9a4078929d72ad9e3 P 66807110b9b54b5b9e7e1ad75b93dcf0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:07:40.409379 31690 raft_consensus.cc:2454] T 4110e82df90741f9a4078929d72ad9e3 P b070fe6b41c6458695df0c6b5e072c50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66807110b9b54b5b9e7e1ad75b93dcf0 in term 0.
I20260809 15:07:40.409510 32223 raft_consensus.cc:482] T 4110e82df90741f9a4078929d72ad9e3 P 66807110b9b54b5b9e7e1ad75b93dcf0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:07:40.409614 32223 raft_consensus.cc:3037] T 4110e82df90741f9a4078929d72ad9e3 P 66807110b9b54b5b9e7e1ad75b93dcf0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:40.411952 32223 raft_consensus.cc:504] T 4110e82df90741f9a4078929d72ad9e3 P 66807110b9b54b5b9e7e1ad75b93dcf0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5192b47149514d4483248c0a10750c00" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 33341 } } peers { permanent_uuid: "66807110b9b54b5b9e7e1ad75b93dcf0" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45377 } } peers { permanent_uuid: "b070fe6b41c6458695df0c6b5e072c50" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 37311 } }
I20260809 15:07:40.412528 32223 leader_election.cc:288] T 4110e82df90741f9a4078929d72ad9e3 P 66807110b9b54b5b9e7e1ad75b93dcf0 [CANDIDATE]: Term 1 election: Requested vote from peers 5192b47149514d4483248c0a10750c00 (127.24.183.194:33341), b070fe6b41c6458695df0c6b5e072c50 (127.24.183.193:37311)
I20260809 15:07:40.412955 31901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4110e82df90741f9a4078929d72ad9e3" candidate_uuid: "66807110b9b54b5b9e7e1ad75b93dcf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5192b47149514d4483248c0a10750c00"
I20260809 15:07:40.413017 31690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4110e82df90741f9a4078929d72ad9e3" candidate_uuid: "66807110b9b54b5b9e7e1ad75b93dcf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b070fe6b41c6458695df0c6b5e072c50"
I20260809 15:07:40.413127 31901 raft_consensus.cc:3037] T 4110e82df90741f9a4078929d72ad9e3 P 5192b47149514d4483248c0a10750c00 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:40.413198 31690 raft_consensus.cc:3037] T 4110e82df90741f9a4078929d72ad9e3 P b070fe6b41c6458695df0c6b5e072c50 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:40.415413 31690 raft_consensus.cc:2454] T 4110e82df90741f9a4078929d72ad9e3 P b070fe6b41c6458695df0c6b5e072c50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66807110b9b54b5b9e7e1ad75b93dcf0 in term 1.
I20260809 15:07:40.415426 31901 raft_consensus.cc:2454] T 4110e82df90741f9a4078929d72ad9e3 P 5192b47149514d4483248c0a10750c00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66807110b9b54b5b9e7e1ad75b93dcf0 in term 1.
I20260809 15:07:40.415827 32007 leader_election.cc:302] T 4110e82df90741f9a4078929d72ad9e3 P 66807110b9b54b5b9e7e1ad75b93dcf0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66807110b9b54b5b9e7e1ad75b93dcf0, b070fe6b41c6458695df0c6b5e072c50; no voters:
I20260809 15:07:40.416110 32223 raft_consensus.cc:2781] T 4110e82df90741f9a4078929d72ad9e3 P 66807110b9b54b5b9e7e1ad75b93dcf0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:07:40.416574 32223 raft_consensus.cc:686] T 4110e82df90741f9a4078929d72ad9e3 P 66807110b9b54b5b9e7e1ad75b93dcf0 [term 1 LEADER]: Becoming Leader. State: Replica: 66807110b9b54b5b9e7e1ad75b93dcf0, State: Running, Role: LEADER
I20260809 15:07:40.417024 32223 consensus_queue.cc:237] T 4110e82df90741f9a4078929d72ad9e3 P 66807110b9b54b5b9e7e1ad75b93dcf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5192b47149514d4483248c0a10750c00" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 33341 } } peers { permanent_uuid: "66807110b9b54b5b9e7e1ad75b93dcf0" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45377 } } peers { permanent_uuid: "b070fe6b41c6458695df0c6b5e072c50" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 37311 } }
I20260809 15:07:40.421648 31470 catalog_manager.cc:5495] T 4110e82df90741f9a4078929d72ad9e3 P 66807110b9b54b5b9e7e1ad75b93dcf0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66807110b9b54b5b9e7e1ad75b93dcf0 (127.24.183.195). New cstate: current_term: 1 leader_uuid: "66807110b9b54b5b9e7e1ad75b93dcf0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5192b47149514d4483248c0a10750c00" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 33341 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66807110b9b54b5b9e7e1ad75b93dcf0" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b070fe6b41c6458695df0c6b5e072c50" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 37311 } health_report { overall_health: UNKNOWN } } }
I20260809 15:07:40.453215 32210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:40.448578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40090 (local address 127.24.183.195:45377)
0809 15:07:40.448752 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:40.448756 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:40.448772 (+ 16us) server_negotiation.cc:407] Connection header received
0809 15:07:40.448848 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:40.448852 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:40.448923 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:40.449045 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:07:40.449325 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:40.450060 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:40.451306 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:40.451717 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:40.451812 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:40.452212 (+ 400us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:07:40.452683 (+ 471us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:07:40.452840 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:40.452889 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 15:07:40.452969 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 15:07:40.460577 31690 raft_consensus.cc:1261] T 4110e82df90741f9a4078929d72ad9e3 P b070fe6b41c6458695df0c6b5e072c50 [term 1 FOLLOWER]: Refusing update from remote peer 66807110b9b54b5b9e7e1ad75b93dcf0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:07:40.460603 31901 raft_consensus.cc:1261] T 4110e82df90741f9a4078929d72ad9e3 P 5192b47149514d4483248c0a10750c00 [term 1 FOLLOWER]: Refusing update from remote peer 66807110b9b54b5b9e7e1ad75b93dcf0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:07:40.461465 32228 consensus_queue.cc:1035] T 4110e82df90741f9a4078929d72ad9e3 P 66807110b9b54b5b9e7e1ad75b93dcf0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b070fe6b41c6458695df0c6b5e072c50" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 37311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:07:40.461793 32223 consensus_queue.cc:1035] T 4110e82df90741f9a4078929d72ad9e3 P 66807110b9b54b5b9e7e1ad75b93dcf0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5192b47149514d4483248c0a10750c00" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 33341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:07:40.471408 32232 mvcc.cc:205] Tried to move back new op lower bound from 7316635895637635072 to 7316635895472144384. Current Snapshot: MvccSnapshot[applied={T|T < 7316635895637635072}]
I20260809 15:07:40.479199 32238 mvcc.cc:205] Tried to move back new op lower bound from 7316635895637635072 to 7316635895472144384. Current Snapshot: MvccSnapshot[applied={T|T < 7316635895637635072}]
I20260809 15:07:40.486413 31470 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:07:40.486670 31470 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:07:40.490152 31470 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c2347b30bbcf4819b5657d75652eaa7a: Sending DeleteTablet for 3 replicas of tablet 4110e82df90741f9a4078929d72ad9e3
I20260809 15:07:40.491504 31661 tablet_service.cc:1514] Processing DeleteTablet for tablet 4110e82df90741f9a4078929d72ad9e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:07:40 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:51568
I20260809 15:07:40.492259 32078 tablet_service.cc:1514] Processing DeleteTablet for tablet 4110e82df90741f9a4078929d72ad9e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:07:40 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:40082
I20260809 15:07:40.492295 31876 tablet_service.cc:1514] Processing DeleteTablet for tablet 4110e82df90741f9a4078929d72ad9e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:07:40 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:54134
I20260809 15:07:40.493191 32248 tablet_replica.cc:330] T 4110e82df90741f9a4078929d72ad9e3 P 5192b47149514d4483248c0a10750c00: stopping tablet replica
I20260809 15:07:40.493353 32246 tablet_replica.cc:330] T 4110e82df90741f9a4078929d72ad9e3 P b070fe6b41c6458695df0c6b5e072c50: stopping tablet replica
I20260809 15:07:40.493640 32248 raft_consensus.cc:2229] T 4110e82df90741f9a4078929d72ad9e3 P 5192b47149514d4483248c0a10750c00 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:07:40.493734 32246 raft_consensus.cc:2229] T 4110e82df90741f9a4078929d72ad9e3 P b070fe6b41c6458695df0c6b5e072c50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:07:40.494168 32248 raft_consensus.cc:2258] T 4110e82df90741f9a4078929d72ad9e3 P 5192b47149514d4483248c0a10750c00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:07:40.494210 32246 raft_consensus.cc:2258] T 4110e82df90741f9a4078929d72ad9e3 P b070fe6b41c6458695df0c6b5e072c50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:07:40.494495 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 31562
I20260809 15:07:40.494921 32247 tablet_replica.cc:330] T 4110e82df90741f9a4078929d72ad9e3 P 66807110b9b54b5b9e7e1ad75b93dcf0: stopping tablet replica
I20260809 15:07:40.495518 32247 raft_consensus.cc:2229] T 4110e82df90741f9a4078929d72ad9e3 P 66807110b9b54b5b9e7e1ad75b93dcf0 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:07:40.496193 32247 raft_consensus.cc:2258] T 4110e82df90741f9a4078929d72ad9e3 P 66807110b9b54b5b9e7e1ad75b93dcf0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:07:40.498268 32247 ts_tablet_manager.cc:1905] T 4110e82df90741f9a4078929d72ad9e3 P 66807110b9b54b5b9e7e1ad75b93dcf0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:07:40.499410 32248 ts_tablet_manager.cc:1905] T 4110e82df90741f9a4078929d72ad9e3 P 5192b47149514d4483248c0a10750c00: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:07:40.500733 32246 ts_tablet_manager.cc:1905] T 4110e82df90741f9a4078929d72ad9e3 P b070fe6b41c6458695df0c6b5e072c50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:07:40.505702 32247 ts_tablet_manager.cc:1918] T 4110e82df90741f9a4078929d72ad9e3 P 66807110b9b54b5b9e7e1ad75b93dcf0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:07:40.505877 32247 log.cc:1196] T 4110e82df90741f9a4078929d72ad9e3 P 66807110b9b54b5b9e7e1ad75b93dcf0: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-2/wal/wals/4110e82df90741f9a4078929d72ad9e3
I20260809 15:07:40.506418 32247 ts_tablet_manager.cc:1939] T 4110e82df90741f9a4078929d72ad9e3 P 66807110b9b54b5b9e7e1ad75b93dcf0: Deleting consensus metadata
I20260809 15:07:40.508163 31450 catalog_manager.cc:4841] TS 66807110b9b54b5b9e7e1ad75b93dcf0 (127.24.183.195:45377): tablet 4110e82df90741f9a4078929d72ad9e3 (table test-table [id=04586c18400f4b90af5704416f42c276]) successfully deleted
I20260809 15:07:40.510346 32248 ts_tablet_manager.cc:1918] T 4110e82df90741f9a4078929d72ad9e3 P 5192b47149514d4483248c0a10750c00: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:07:40.510515 32248 log.cc:1196] T 4110e82df90741f9a4078929d72ad9e3 P 5192b47149514d4483248c0a10750c00: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-1/wal/wals/4110e82df90741f9a4078929d72ad9e3
I20260809 15:07:40.511070 32248 ts_tablet_manager.cc:1939] T 4110e82df90741f9a4078929d72ad9e3 P 5192b47149514d4483248c0a10750c00: Deleting consensus metadata
Aug 09 15:07:40 dist-test-slave-nj21 krb5kdc[31408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.193@KRBTEST.COM
I20260809 15:07:40.512823 31450 catalog_manager.cc:4841] TS 5192b47149514d4483248c0a10750c00 (127.24.183.194:33341): tablet 4110e82df90741f9a4078929d72ad9e3 (table test-table [id=04586c18400f4b90af5704416f42c276]) successfully deleted
I20260809 15:07:40.513242 32246 ts_tablet_manager.cc:1918] T 4110e82df90741f9a4078929d72ad9e3 P b070fe6b41c6458695df0c6b5e072c50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:07:40.513415 32246 log.cc:1196] T 4110e82df90741f9a4078929d72ad9e3 P b070fe6b41c6458695df0c6b5e072c50: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288021313853-25311-0/minicluster-data/ts-0/wal/wals/4110e82df90741f9a4078929d72ad9e3
I20260809 15:07:40.513882 32246 ts_tablet_manager.cc:1939] T 4110e82df90741f9a4078929d72ad9e3 P b070fe6b41c6458695df0c6b5e072c50: Deleting consensus metadata
I20260809 15:07:40.515385 31451 catalog_manager.cc:4841] TS b070fe6b41c6458695df0c6b5e072c50 (127.24.183.193:37311): tablet 4110e82df90741f9a4078929d72ad9e3 (table test-table [id=04586c18400f4b90af5704416f42c276]) successfully deleted
I20260809 15:07:40.520582 32209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:40.495509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51584 (local address 127.24.183.193:37311)
0809 15:07:40.495709 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:40.495716 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:40.502574 (+ 6858us) server_negotiation.cc:407] Connection header received
0809 15:07:40.502762 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:40.502768 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:40.502855 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:40.502949 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:40.504645 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:40.505718 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:40.507561 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:40.508229 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:40.508336 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:40.513751 (+ 5415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:40.513766 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:40.513786 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:40.513836 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:40.517682 (+ 3846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:40.518368 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:40.518377 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:40.518380 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:40.518546 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:40.518922 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:40.518927 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:40.518930 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:40.519416 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:40.519751 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:40.519997 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 15:07:40.520223 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 15:07:40.521238 31745 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:07:40.610944 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 31562
I20260809 15:07:40.637629 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 31766
Aug 09 15:07:40 dist-test-slave-nj21 krb5kdc[31408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.194@KRBTEST.COM
I20260809 15:07:40.652374 32204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:40.638424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54138 (local address 127.24.183.194:33341)
0809 15:07:40.638619 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:40.638626 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:40.638731 (+ 105us) server_negotiation.cc:407] Connection header received
0809 15:07:40.638916 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:40.638921 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:40.638989 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:40.639092 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:40.640404 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:40.641167 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:40.642358 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:40.642751 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:40.642868 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:40.646804 (+ 3936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:40.646820 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:40.646838 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:40.646871 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:40.649696 (+ 2825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:40.650327 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:40.650334 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:40.650337 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:40.650460 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:40.650803 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:40.650808 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:40.650810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:40.651190 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:40.651478 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:40.651778 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 15:07:40.652008 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 15:07:40.653030 31963 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:07:40.743700 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 31766
I20260809 15:07:40.768711 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 31975
Aug 09 15:07:40 dist-test-slave-nj21 krb5kdc[31408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.195@KRBTEST.COM
I20260809 15:07:40.785657 32210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:40.769509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40096 (local address 127.24.183.195:45377)
0809 15:07:40.769682 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:40.769686 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:40.769710 (+ 24us) server_negotiation.cc:407] Connection header received
0809 15:07:40.769929 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:40.769934 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:40.770005 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:40.770086 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:40.771898 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:40.772736 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:40.774548 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:40.775029 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:40.775167 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:40.779733 (+ 4566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:40.779745 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:40.779764 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:40.779811 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:40.782732 (+ 2921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:40.783517 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:40.783528 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:40.783530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:40.783741 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:40.784155 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:40.784158 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:40.784160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:40.784552 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:40.784842 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:40.785149 (+ 307us) server_negotiation.cc:299] Negotiation successful
0809 15:07:40.785312 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 15:07:40.786374 32172 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:07:40.909885 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 31975
I20260809 15:07:40.933717 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 31428
I20260809 15:07:40.934957 31524 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:07:41.046989 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 31428
2026-08-09T15:07:41Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4276 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T15:07:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:07:41Z Disabled control of system clock
I20260809 15:07:41.121654 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:42993
--webserver_interface=127.24.183.254
--webserver_port=0
--builtin_ntp_servers=127.24.183.212:32819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:42993
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 15:07:41.444808 32267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:41.445075 32267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:41.445168 32267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:41.452135 32267 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:07:41.452229 32267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:41.452273 32267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:41.452337 32267 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:07:41.452380 32267 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:07:41.461097 32267 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:32819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:42993
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:42993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:41.462271 32267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:41.463999 32267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:41.475024 32276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:41.475113 32274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:41.475214 32272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:41.476060 32267 server_base.cc:1034] running on GCE node
I20260809 15:07:41.476781 32267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:41.478029 32267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:41.479229 32267 hybrid_clock.cc:648] HybridClock initialized: now 1786288061479179 us; error 40 us; skew 500 ppm
I20260809 15:07:41.479553 32267 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:07:41.481627 32267 webserver.cc:460] Webserver started at http://127.24.183.254:36675/ using document root <none> and password file <none>
I20260809 15:07:41.482272 32267 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:41.482378 32267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:41.482688 32267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:41.485446 32267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "179db7c38112497ca6091221cfd3886f"
format_stamp: "Formatted at 2026-08-09 15:07:41 on dist-test-slave-nj21"
server_key: "dcf591517f98ddcc684589e343009743"
server_key_iv: "b80713d9757e1d909f0c0fc1af72881f"
server_key_version: "encryptionkey@0"
I20260809 15:07:41.486191 32267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "179db7c38112497ca6091221cfd3886f"
format_stamp: "Formatted at 2026-08-09 15:07:41 on dist-test-slave-nj21"
server_key: "dcf591517f98ddcc684589e343009743"
server_key_iv: "b80713d9757e1d909f0c0fc1af72881f"
server_key_version: "encryptionkey@0"
I20260809 15:07:41.490976 32267 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 15:07:41.494160 32285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:41.495361 32267 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:07:41.495508 32267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "179db7c38112497ca6091221cfd3886f"
format_stamp: "Formatted at 2026-08-09 15:07:41 on dist-test-slave-nj21"
server_key: "dcf591517f98ddcc684589e343009743"
server_key_iv: "b80713d9757e1d909f0c0fc1af72881f"
server_key_version: "encryptionkey@0"
I20260809 15:07:41.495714 32267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:41.501071 32267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:41.501767 32267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:41.502031 32267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:41.519168 32267 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:42993
I20260809 15:07:41.519204 32360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:42993 every 8 connection(s)
I20260809 15:07:41.520530 32267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:07:41.524072 32363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:41.529778 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 32267
I20260809 15:07:41.529963 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:07:41.530609 25311 external_mini_cluster.cc:1428] Setting key f6dfbb7b55b2f7e6426fa3c9692abd69
I20260809 15:07:41.534945 32363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 179db7c38112497ca6091221cfd3886f: Bootstrap starting.
I20260809 15:07:41.538936 32363 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 179db7c38112497ca6091221cfd3886f: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:41.539846 32367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:41.532586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51482 (local address 127.24.183.254:42993)
0809 15:07:41.533607 (+ 1021us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:41.533622 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:41.533672 (+ 50us) server_negotiation.cc:407] Connection header received
0809 15:07:41.535950 (+ 2278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:41.535977 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:41.536452 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:41.536923 (+ 471us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:41.537939 (+ 1016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:41.537948 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:41.537964 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:07:41.537983 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:41.538055 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:07:41.538080 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:41.538253 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:41.538374 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 15:07:41.538620 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":309,"threads_started":1}
I20260809 15:07:41.540297 32363 log.cc:824] T 00000000000000000000000000000000 P 179db7c38112497ca6091221cfd3886f: Log is configured to *not* fsync() on all Append() calls
I20260809 15:07:41.543299 32363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 179db7c38112497ca6091221cfd3886f: No bootstrap required, opened a new log
I20260809 15:07:41.550554 32363 raft_consensus.cc:348] T 00000000000000000000000000000000 P 179db7c38112497ca6091221cfd3886f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "179db7c38112497ca6091221cfd3886f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 42993 } }
I20260809 15:07:41.550872 32363 raft_consensus.cc:374] T 00000000000000000000000000000000 P 179db7c38112497ca6091221cfd3886f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:41.550974 32363 raft_consensus.cc:729] T 00000000000000000000000000000000 P 179db7c38112497ca6091221cfd3886f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 179db7c38112497ca6091221cfd3886f, State: Initialized, Role: FOLLOWER
I20260809 15:07:41.551473 32363 consensus_queue.cc:260] T 00000000000000000000000000000000 P 179db7c38112497ca6091221cfd3886f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "179db7c38112497ca6091221cfd3886f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 42993 } }
I20260809 15:07:41.551715 32363 raft_consensus.cc:388] T 00000000000000000000000000000000 P 179db7c38112497ca6091221cfd3886f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:07:41.551851 32363 raft_consensus.cc:482] T 00000000000000000000000000000000 P 179db7c38112497ca6091221cfd3886f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:07:41.552007 32363 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 179db7c38112497ca6091221cfd3886f [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:41.554251 32363 raft_consensus.cc:504] T 00000000000000000000000000000000 P 179db7c38112497ca6091221cfd3886f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "179db7c38112497ca6091221cfd3886f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 42993 } }
I20260809 15:07:41.554715 32363 leader_election.cc:302] T 00000000000000000000000000000000 P 179db7c38112497ca6091221cfd3886f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 179db7c38112497ca6091221cfd3886f; no voters:
I20260809 15:07:41.555259 32363 leader_election.cc:288] T 00000000000000000000000000000000 P 179db7c38112497ca6091221cfd3886f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:07:41.555379 32370 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 179db7c38112497ca6091221cfd3886f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:07:41.556068 32370 raft_consensus.cc:686] T 00000000000000000000000000000000 P 179db7c38112497ca6091221cfd3886f [term 1 LEADER]: Becoming Leader. State: Replica: 179db7c38112497ca6091221cfd3886f, State: Running, Role: LEADER
I20260809 15:07:41.556568 32370 consensus_queue.cc:237] T 00000000000000000000000000000000 P 179db7c38112497ca6091221cfd3886f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "179db7c38112497ca6091221cfd3886f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 42993 } }
I20260809 15:07:41.557209 32363 sys_catalog.cc:564] T 00000000000000000000000000000000 P 179db7c38112497ca6091221cfd3886f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:07:41.559033 32372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 179db7c38112497ca6091221cfd3886f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 179db7c38112497ca6091221cfd3886f. Latest consensus state: current_term: 1 leader_uuid: "179db7c38112497ca6091221cfd3886f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "179db7c38112497ca6091221cfd3886f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 42993 } } }
I20260809 15:07:41.559450 32372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 179db7c38112497ca6091221cfd3886f [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:41.559095 32371 sys_catalog.cc:455] T 00000000000000000000000000000000 P 179db7c38112497ca6091221cfd3886f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "179db7c38112497ca6091221cfd3886f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "179db7c38112497ca6091221cfd3886f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 42993 } } }
I20260809 15:07:41.559764 32371 sys_catalog.cc:458] T 00000000000000000000000000000000 P 179db7c38112497ca6091221cfd3886f [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:41.561475 32385 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:07:41.565608 32385 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:07:41.573593 32385 catalog_manager.cc:1347] Generated new cluster ID: 9b558e4f50a94d9b80ebe413163286d3
I20260809 15:07:41.573700 32385 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:07:41.607796 32385 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:07:41.608281 32385 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:07:41.614714 32385 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 179db7c38112497ca6091221cfd3886f: Generated new TSK 0
I20260809 15:07:41.615456 32385 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:07:41.623334 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.193:0
--local_ip_for_outbound_sockets=127.24.183.193
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:42993
--builtin_ntp_servers=127.24.183.212:32819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 15:07:41.963388 32404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:41.963639 32404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:41.963792 32404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:41.970847 32404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:41.970940 32404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:41.971087 32404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.193
I20260809 15:07:41.979861 32404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:32819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:42993
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.193
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:41.981009 32404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:41.982723 32404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:41.995249 32413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:41.995419 32412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:41.995275 32416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:41.995991 32404 server_base.cc:1034] running on GCE node
I20260809 15:07:41.996589 32404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:41.997447 32404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:41.998673 32404 hybrid_clock.cc:648] HybridClock initialized: now 1786288061998605 us; error 42 us; skew 500 ppm
I20260809 15:07:41.999003 32404 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:07:42.001144 32404 webserver.cc:460] Webserver started at http://127.24.183.193:46593/ using document root <none> and password file <none>
I20260809 15:07:42.001770 32404 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:42.001869 32404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:42.002177 32404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:42.004978 32404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-0/data/instance:
uuid: "b5999f170a954efe8fd8a2c9cb624dac"
format_stamp: "Formatted at 2026-08-09 15:07:41 on dist-test-slave-nj21"
server_key: "fcd2cf8f17b4bac16877520ffb2b8d4c"
server_key_iv: "ca4c9e71ecfbe3ff6951e62c46d24350"
server_key_version: "encryptionkey@0"
I20260809 15:07:42.005784 32404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance:
uuid: "b5999f170a954efe8fd8a2c9cb624dac"
format_stamp: "Formatted at 2026-08-09 15:07:41 on dist-test-slave-nj21"
server_key: "fcd2cf8f17b4bac16877520ffb2b8d4c"
server_key_iv: "ca4c9e71ecfbe3ff6951e62c46d24350"
server_key_version: "encryptionkey@0"
I20260809 15:07:42.010525 32404 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260809 15:07:42.013741 32429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:42.015007 32404 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 15:07:42.015164 32404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-0/wal
uuid: "b5999f170a954efe8fd8a2c9cb624dac"
format_stamp: "Formatted at 2026-08-09 15:07:41 on dist-test-slave-nj21"
server_key: "fcd2cf8f17b4bac16877520ffb2b8d4c"
server_key_iv: "ca4c9e71ecfbe3ff6951e62c46d24350"
server_key_version: "encryptionkey@0"
I20260809 15:07:42.015388 32404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:42.020772 32404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:42.021536 32404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:42.021780 32404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:42.022627 32404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:07:42.024001 32404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:07:42.024096 32404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:42.024202 32404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:07:42.024272 32404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:42.054682 32404 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.193:42347
I20260809 15:07:42.054744 32596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.193:42347 every 8 connection(s)
I20260809 15:07:42.056010 32404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
I20260809 15:07:42.064118 32601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:42.058594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:34449 (local address 127.24.183.254:42993)
0809 15:07:42.058987 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:42.058991 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:42.060209 (+ 1218us) server_negotiation.cc:407] Connection header received
0809 15:07:42.061971 (+ 1762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:42.061976 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:42.062045 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:42.062144 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:42.063333 (+ 1189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:42.063339 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:42.063344 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:07:42.063360 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:42.063405 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:07:42.063417 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:42.063526 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:42.063733 (+ 207us) server_negotiation.cc:299] Negotiation successful
0809 15:07:42.063805 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":179,"threads_started":1}
I20260809 15:07:42.064936 32603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:42.059149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:42993 (local address 127.24.183.193:34449)
0809 15:07:42.060058 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:42.060100 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:42.061657 (+ 1557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:42.062381 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:42.062389 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:42.062472 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:42.063047 (+ 575us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:07:42.063050 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:42.063085 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:07:42.063168 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:42.063508 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:42.063518 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:42.063526 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 15:07:42.063800 (+ 274us) client_negotiation.cc:240] Negotiation successful
0809 15:07:42.064038 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":262,"threads_started":1}
I20260809 15:07:42.066242 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 32404
I20260809 15:07:42.066298 32597 heartbeater.cc:347] Connected to a master server at 127.24.183.254:42993
I20260809 15:07:42.066453 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance
I20260809 15:07:42.066619 32597 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:42.067288 25311 external_mini_cluster.cc:1428] Setting key d6f8e5a53d9e90eb425d7825d101a766
I20260809 15:07:42.067322 32597 heartbeater.cc:511] Master 127.24.183.254:42993 requested a full tablet report, sending...
I20260809 15:07:42.069510 32311 ts_manager.cc:194] Registered new tserver with Master: b5999f170a954efe8fd8a2c9cb624dac (127.24.183.193:42347)
I20260809 15:07:42.070058 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.194:0
--local_ip_for_outbound_sockets=127.24.183.194
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:42993
--builtin_ntp_servers=127.24.183.212:32819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 15:07:42.395589 32604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:42.395888 32604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:42.396020 32604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:42.403290 32604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:42.403402 32604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:42.403558 32604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.194
I20260809 15:07:42.412526 32604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:32819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:42993
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.194
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:42.413754 32604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:42.415490 32604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:42.427989 32611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:42.428153 32612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:42.428139 32615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:42.428970 32604 server_base.cc:1034] running on GCE node
I20260809 15:07:42.429483 32604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:42.430303 32604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:42.431500 32604 hybrid_clock.cc:648] HybridClock initialized: now 1786288062431458 us; error 32 us; skew 500 ppm
I20260809 15:07:42.431816 32604 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:07:42.433844 32604 webserver.cc:460] Webserver started at http://127.24.183.194:34649/ using document root <none> and password file <none>
I20260809 15:07:42.434469 32604 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:42.434563 32604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:42.434870 32604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:42.437588 32604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-1/data/instance:
uuid: "99ced08fd0bb4678b13a4dde54b67b03"
format_stamp: "Formatted at 2026-08-09 15:07:42 on dist-test-slave-nj21"
server_key: "3f205c5ac0cfec33671b8890fa9aeef4"
server_key_iv: "1e6adcc7012e32e2f36a0288644852cc"
server_key_version: "encryptionkey@0"
I20260809 15:07:42.438319 32604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance:
uuid: "99ced08fd0bb4678b13a4dde54b67b03"
format_stamp: "Formatted at 2026-08-09 15:07:42 on dist-test-slave-nj21"
server_key: "3f205c5ac0cfec33671b8890fa9aeef4"
server_key_iv: "1e6adcc7012e32e2f36a0288644852cc"
server_key_version: "encryptionkey@0"
I20260809 15:07:42.443058 32604 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260809 15:07:42.446236 32624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:42.447396 32604 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 15:07:42.447536 32604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-1/wal
uuid: "99ced08fd0bb4678b13a4dde54b67b03"
format_stamp: "Formatted at 2026-08-09 15:07:42 on dist-test-slave-nj21"
server_key: "3f205c5ac0cfec33671b8890fa9aeef4"
server_key_iv: "1e6adcc7012e32e2f36a0288644852cc"
server_key_version: "encryptionkey@0"
I20260809 15:07:42.447760 32604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:42.453521 32604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:42.454236 32604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:42.454484 32604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:42.455323 32604 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:07:42.456720 32604 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:07:42.456815 32604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:42.456920 32604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:07:42.456992 32604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:42.487936 32604 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.194:36791
I20260809 15:07:42.487987 342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.194:36791 every 8 connection(s)
I20260809 15:07:42.489194 32604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
I20260809 15:07:42.490053 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 32604
I20260809 15:07:42.490217 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance
I20260809 15:07:42.490794 25311 external_mini_cluster.cc:1428] Setting key 150a7670eae5c6194d31a2bad0b0c4de
I20260809 15:07:42.494431 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.195:0
--local_ip_for_outbound_sockets=127.24.183.195
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:42993
--builtin_ntp_servers=127.24.183.212:32819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 15:07:42.498301 32601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:42.492064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:55685 (local address 127.24.183.254:42993)
0809 15:07:42.492213 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:42.492217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:42.493912 (+ 1695us) server_negotiation.cc:407] Connection header received
0809 15:07:42.496200 (+ 2288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:42.496204 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:42.496255 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:42.496337 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:42.497649 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:42.497653 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:42.497656 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:07:42.497672 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:42.497706 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:07:42.497717 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:42.497811 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:42.497997 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 15:07:42.498046 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 15:07:42.499475 346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:42.492671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:42993 (local address 127.24.183.194:55685)
0809 15:07:42.493750 (+ 1079us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:42.493796 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:42.495933 (+ 2137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:42.496561 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:42.496572 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:42.496680 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:42.497328 (+ 648us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:07:42.497331 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:42.497377 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:07:42.497482 (+ 105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:42.497795 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:42.497804 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:42.497811 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 15:07:42.498073 (+ 262us) client_negotiation.cc:240] Negotiation successful
0809 15:07:42.498312 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":339,"threads_started":1}
I20260809 15:07:42.500717 343 heartbeater.cc:347] Connected to a master server at 127.24.183.254:42993
I20260809 15:07:42.501019 343 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:42.501669 343 heartbeater.cc:511] Master 127.24.183.254:42993 requested a full tablet report, sending...
I20260809 15:07:42.503100 32311 ts_manager.cc:194] Registered new tserver with Master: 99ced08fd0bb4678b13a4dde54b67b03 (127.24.183.194:36791)
W20260809 15:07:42.823717 347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:42.824002 347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:42.824131 347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:42.831306 347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:42.831403 347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:42.831578 347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.195
I20260809 15:07:42.840569 347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:32819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:42993
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.195
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:42.841872 347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:42.843662 347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:42.856417 360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:42.856375 365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:42.856336 362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:42.857259 347 server_base.cc:1034] running on GCE node
I20260809 15:07:42.857829 347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:42.858716 347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:42.859925 347 hybrid_clock.cc:648] HybridClock initialized: now 1786288062859859 us; error 41 us; skew 500 ppm
I20260809 15:07:42.860224 347 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:07:42.862370 347 webserver.cc:460] Webserver started at http://127.24.183.195:40201/ using document root <none> and password file <none>
I20260809 15:07:42.863008 347 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:42.863107 347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:42.863437 347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:42.866255 347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-2/data/instance:
uuid: "f00b23359dc141bd9b6e3794019c4d4d"
format_stamp: "Formatted at 2026-08-09 15:07:42 on dist-test-slave-nj21"
server_key: "c25689f04befa48a0b33ca533ab2e432"
server_key_iv: "671552c79afb43f89b341d3c41652e6c"
server_key_version: "encryptionkey@0"
I20260809 15:07:42.867022 347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance:
uuid: "f00b23359dc141bd9b6e3794019c4d4d"
format_stamp: "Formatted at 2026-08-09 15:07:42 on dist-test-slave-nj21"
server_key: "c25689f04befa48a0b33ca533ab2e432"
server_key_iv: "671552c79afb43f89b341d3c41652e6c"
server_key_version: "encryptionkey@0"
I20260809 15:07:42.871904 347 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260809 15:07:42.875097 378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:42.876370 347 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:07:42.876510 347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-2/wal
uuid: "f00b23359dc141bd9b6e3794019c4d4d"
format_stamp: "Formatted at 2026-08-09 15:07:42 on dist-test-slave-nj21"
server_key: "c25689f04befa48a0b33ca533ab2e432"
server_key_iv: "671552c79afb43f89b341d3c41652e6c"
server_key_version: "encryptionkey@0"
I20260809 15:07:42.876699 347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:42.882086 347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:42.882833 347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:42.883085 347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:42.883982 347 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:07:42.885334 347 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:07:42.885429 347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:42.885532 347 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:07:42.885601 347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:42.915491 347 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.195:32937
I20260809 15:07:42.915535 566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.195:32937 every 8 connection(s)
I20260809 15:07:42.916785 347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
I20260809 15:07:42.924307 32601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:42.919372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:50205 (local address 127.24.183.254:42993)
0809 15:07:42.919576 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:42.919583 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:42.920922 (+ 1339us) server_negotiation.cc:407] Connection header received
0809 15:07:42.922514 (+ 1592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:42.922518 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:42.922572 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:42.922657 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:42.923739 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:42.923744 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:42.923747 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:07:42.923762 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:42.923794 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:07:42.923805 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:42.923899 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:42.924039 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 15:07:42.924089 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 15:07:42.925136 575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:42.919912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:42993 (local address 127.24.183.195:50205)
0809 15:07:42.920780 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:42.920815 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:42.922285 (+ 1470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:42.922822 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:42.922829 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:42.922909 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:42.923454 (+ 545us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:07:42.923457 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:42.923492 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:07:42.923569 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:42.923881 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:42.923888 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:42.923893 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 15:07:42.924095 (+ 202us) client_negotiation.cc:240] Negotiation successful
0809 15:07:42.924268 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":259,"threads_started":1}
I20260809 15:07:42.926388 569 heartbeater.cc:347] Connected to a master server at 127.24.183.254:42993
I20260809 15:07:42.926647 569 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:42.926882 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 347
I20260809 15:07:42.927018 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance
I20260809 15:07:42.927237 569 heartbeater.cc:511] Master 127.24.183.254:42993 requested a full tablet report, sending...
I20260809 15:07:42.927477 25311 external_mini_cluster.cc:1428] Setting key e87ca3da61c58ea02119e0791098ce18
I20260809 15:07:42.928622 32311 ts_manager.cc:194] Registered new tserver with Master: f00b23359dc141bd9b6e3794019c4d4d (127.24.183.195:32937)
I20260809 15:07:42.930267 25311 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:07:42.936702 32601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:42.934485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51488 (local address 127.24.183.254:42993)
0809 15:07:42.934633 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:42.934637 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:42.935161 (+ 524us) server_negotiation.cc:407] Connection header received
0809 15:07:42.935277 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:42.935281 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:42.935329 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:42.935409 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:42.936187 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:42.936191 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:42.936194 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:07:42.936208 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:42.936238 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:07:42.936249 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:42.936341 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:42.936400 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 15:07:42.936444 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 15:07:42.940279 32311 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51488:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:07:42.947324 32311 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:07:42.963797 588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:42.959378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56224 (local address 127.24.183.193:42347)
0809 15:07:42.959993 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:42.960002 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:42.960036 (+ 34us) server_negotiation.cc:407] Connection header received
0809 15:07:42.960250 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:42.960260 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:42.960524 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:42.960693 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:42.962873 (+ 2180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:42.962883 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:42.962898 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:07:42.962927 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:42.963005 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:07:42.963026 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:42.963156 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:42.963309 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 15:07:42.963422 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":312,"threads_started":1}
I20260809 15:07:42.963965 589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:42.960245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:32937 (local address 127.0.0.1:57190)
0809 15:07:42.960926 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:42.960951 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:42.961052 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:42.962042 (+ 990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:42.962046 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:42.962069 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:42.962670 (+ 601us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:07:42.962673 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:42.962690 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:07:42.962735 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:42.963187 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:42.963191 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:42.963196 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 15:07:42.963321 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 15:07:42.963445 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"mutex_wait_us":26,"thread_start_us":426,"threads_started":1}
I20260809 15:07:42.964447 592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:42.960537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57190 (local address 127.24.183.195:32937)
0809 15:07:42.961656 (+ 1119us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:42.961664 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:42.961694 (+ 30us) server_negotiation.cc:407] Connection header received
0809 15:07:42.961749 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:42.961754 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:42.961894 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:42.962021 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:42.962996 (+ 975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:42.963003 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:42.963012 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:07:42.963024 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:42.963058 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:07:42.963071 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:42.963873 (+ 802us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:42.963945 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 15:07:42.964037 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":955,"thread_start_us":783,"threads_started":1}
I20260809 15:07:42.964493 587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:42.959142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:42347 (local address 127.0.0.1:56224)
0809 15:07:42.959918 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:42.959962 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:42.960105 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:42.961708 (+ 1603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:42.961716 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:42.961813 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:42.962552 (+ 739us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:07:42.962556 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:42.962586 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:07:42.962684 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:42.963128 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:42.963140 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:42.963149 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 15:07:42.963345 (+ 196us) client_negotiation.cc:240] Negotiation successful
0809 15:07:42.963429 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":284,"threads_started":1}
I20260809 15:07:42.966645 589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:42.961004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:36791 (local address 127.0.0.1:34148)
0809 15:07:42.964157 (+ 3153us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:42.964172 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:42.964254 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:42.965318 (+ 1064us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:42.965323 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:42.965344 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:42.965887 (+ 543us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:07:42.965890 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:42.965905 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:07:42.965947 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:42.966275 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:42.966279 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:42.966284 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 15:07:42.966367 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 15:07:42.966414 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3073,"threads_started":1}
I20260809 15:07:42.966799 595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:42.961509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34148 (local address 127.24.183.194:36791)
0809 15:07:42.964963 (+ 3454us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:42.964969 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:42.964990 (+ 21us) server_negotiation.cc:407] Connection header received
0809 15:07:42.965043 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:42.965048 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:42.965177 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:42.965300 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:42.966110 (+ 810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:42.966115 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:42.966124 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:07:42.966136 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:42.966170 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:07:42.966188 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:42.966341 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:42.966399 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 15:07:42.966489 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3329,"thread_start_us":3220,"threads_started":1}
I20260809 15:07:42.970593 32501 tablet_service.cc:1467] Processing CreateTablet for tablet c2d3b40ce1944fd7b455cdf935bf438c (DEFAULT_TABLE table=test-table [id=40c5c61fbe2f4643a112a93ad4d7a0f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:07:42.970927 32704 tablet_service.cc:1467] Processing CreateTablet for tablet c2d3b40ce1944fd7b455cdf935bf438c (DEFAULT_TABLE table=test-table [id=40c5c61fbe2f4643a112a93ad4d7a0f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:07:42.970927 456 tablet_service.cc:1467] Processing CreateTablet for tablet c2d3b40ce1944fd7b455cdf935bf438c (DEFAULT_TABLE table=test-table [id=40c5c61fbe2f4643a112a93ad4d7a0f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:07:42.972149 32501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2d3b40ce1944fd7b455cdf935bf438c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:42.972281 456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2d3b40ce1944fd7b455cdf935bf438c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:42.972281 32704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2d3b40ce1944fd7b455cdf935bf438c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:42.984625 599 tablet_bootstrap.cc:492] T c2d3b40ce1944fd7b455cdf935bf438c P b5999f170a954efe8fd8a2c9cb624dac: Bootstrap starting.
I20260809 15:07:42.986274 600 tablet_bootstrap.cc:492] T c2d3b40ce1944fd7b455cdf935bf438c P f00b23359dc141bd9b6e3794019c4d4d: Bootstrap starting.
I20260809 15:07:42.987082 601 tablet_bootstrap.cc:492] T c2d3b40ce1944fd7b455cdf935bf438c P 99ced08fd0bb4678b13a4dde54b67b03: Bootstrap starting.
I20260809 15:07:42.988045 599 tablet_bootstrap.cc:654] T c2d3b40ce1944fd7b455cdf935bf438c P b5999f170a954efe8fd8a2c9cb624dac: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:42.989336 599 log.cc:824] T c2d3b40ce1944fd7b455cdf935bf438c P b5999f170a954efe8fd8a2c9cb624dac: Log is configured to *not* fsync() on all Append() calls
I20260809 15:07:42.990105 601 tablet_bootstrap.cc:654] T c2d3b40ce1944fd7b455cdf935bf438c P 99ced08fd0bb4678b13a4dde54b67b03: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:42.990566 600 tablet_bootstrap.cc:654] T c2d3b40ce1944fd7b455cdf935bf438c P f00b23359dc141bd9b6e3794019c4d4d: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:42.991097 601 log.cc:824] T c2d3b40ce1944fd7b455cdf935bf438c P 99ced08fd0bb4678b13a4dde54b67b03: Log is configured to *not* fsync() on all Append() calls
I20260809 15:07:42.991833 600 log.cc:824] T c2d3b40ce1944fd7b455cdf935bf438c P f00b23359dc141bd9b6e3794019c4d4d: Log is configured to *not* fsync() on all Append() calls
I20260809 15:07:42.993767 601 tablet_bootstrap.cc:492] T c2d3b40ce1944fd7b455cdf935bf438c P 99ced08fd0bb4678b13a4dde54b67b03: No bootstrap required, opened a new log
I20260809 15:07:42.993993 601 ts_tablet_manager.cc:1397] T c2d3b40ce1944fd7b455cdf935bf438c P 99ced08fd0bb4678b13a4dde54b67b03: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260809 15:07:42.994343 600 tablet_bootstrap.cc:492] T c2d3b40ce1944fd7b455cdf935bf438c P f00b23359dc141bd9b6e3794019c4d4d: No bootstrap required, opened a new log
I20260809 15:07:42.994567 600 ts_tablet_manager.cc:1397] T c2d3b40ce1944fd7b455cdf935bf438c P f00b23359dc141bd9b6e3794019c4d4d: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 15:07:42.997581 599 tablet_bootstrap.cc:492] T c2d3b40ce1944fd7b455cdf935bf438c P b5999f170a954efe8fd8a2c9cb624dac: No bootstrap required, opened a new log
I20260809 15:07:42.997833 599 ts_tablet_manager.cc:1397] T c2d3b40ce1944fd7b455cdf935bf438c P b5999f170a954efe8fd8a2c9cb624dac: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.000s
I20260809 15:07:43.001528 601 raft_consensus.cc:348] T c2d3b40ce1944fd7b455cdf935bf438c P 99ced08fd0bb4678b13a4dde54b67b03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5999f170a954efe8fd8a2c9cb624dac" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 42347 } } peers { permanent_uuid: "f00b23359dc141bd9b6e3794019c4d4d" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 32937 } } peers { permanent_uuid: "99ced08fd0bb4678b13a4dde54b67b03" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 36791 } }
I20260809 15:07:43.001857 601 raft_consensus.cc:374] T c2d3b40ce1944fd7b455cdf935bf438c P 99ced08fd0bb4678b13a4dde54b67b03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:43.001981 601 raft_consensus.cc:729] T c2d3b40ce1944fd7b455cdf935bf438c P 99ced08fd0bb4678b13a4dde54b67b03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99ced08fd0bb4678b13a4dde54b67b03, State: Initialized, Role: FOLLOWER
I20260809 15:07:43.002527 601 consensus_queue.cc:260] T c2d3b40ce1944fd7b455cdf935bf438c P 99ced08fd0bb4678b13a4dde54b67b03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5999f170a954efe8fd8a2c9cb624dac" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 42347 } } peers { permanent_uuid: "f00b23359dc141bd9b6e3794019c4d4d" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 32937 } } peers { permanent_uuid: "99ced08fd0bb4678b13a4dde54b67b03" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 36791 } }
I20260809 15:07:43.003439 343 heartbeater.cc:503] Master 127.24.183.254:42993 was elected leader, sending a full tablet report...
I20260809 15:07:43.004101 601 ts_tablet_manager.cc:1428] T c2d3b40ce1944fd7b455cdf935bf438c P 99ced08fd0bb4678b13a4dde54b67b03: Time spent starting tablet: real 0.010s user 0.004s sys 0.007s
I20260809 15:07:43.004066 600 raft_consensus.cc:348] T c2d3b40ce1944fd7b455cdf935bf438c P f00b23359dc141bd9b6e3794019c4d4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5999f170a954efe8fd8a2c9cb624dac" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 42347 } } peers { permanent_uuid: "f00b23359dc141bd9b6e3794019c4d4d" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 32937 } } peers { permanent_uuid: "99ced08fd0bb4678b13a4dde54b67b03" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 36791 } }
I20260809 15:07:43.004458 600 raft_consensus.cc:374] T c2d3b40ce1944fd7b455cdf935bf438c P f00b23359dc141bd9b6e3794019c4d4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:43.004582 600 raft_consensus.cc:729] T c2d3b40ce1944fd7b455cdf935bf438c P f00b23359dc141bd9b6e3794019c4d4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00b23359dc141bd9b6e3794019c4d4d, State: Initialized, Role: FOLLOWER
I20260809 15:07:43.005298 600 consensus_queue.cc:260] T c2d3b40ce1944fd7b455cdf935bf438c P f00b23359dc141bd9b6e3794019c4d4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5999f170a954efe8fd8a2c9cb624dac" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 42347 } } peers { permanent_uuid: "f00b23359dc141bd9b6e3794019c4d4d" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 32937 } } peers { permanent_uuid: "99ced08fd0bb4678b13a4dde54b67b03" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 36791 } }
I20260809 15:07:43.006454 569 heartbeater.cc:503] Master 127.24.183.254:42993 was elected leader, sending a full tablet report...
I20260809 15:07:43.007005 600 ts_tablet_manager.cc:1428] T c2d3b40ce1944fd7b455cdf935bf438c P f00b23359dc141bd9b6e3794019c4d4d: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260809 15:07:43.007952 599 raft_consensus.cc:348] T c2d3b40ce1944fd7b455cdf935bf438c P b5999f170a954efe8fd8a2c9cb624dac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5999f170a954efe8fd8a2c9cb624dac" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 42347 } } peers { permanent_uuid: "f00b23359dc141bd9b6e3794019c4d4d" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 32937 } } peers { permanent_uuid: "99ced08fd0bb4678b13a4dde54b67b03" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 36791 } }
I20260809 15:07:43.008262 599 raft_consensus.cc:374] T c2d3b40ce1944fd7b455cdf935bf438c P b5999f170a954efe8fd8a2c9cb624dac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:43.008365 599 raft_consensus.cc:729] T c2d3b40ce1944fd7b455cdf935bf438c P b5999f170a954efe8fd8a2c9cb624dac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5999f170a954efe8fd8a2c9cb624dac, State: Initialized, Role: FOLLOWER
I20260809 15:07:43.009073 599 consensus_queue.cc:260] T c2d3b40ce1944fd7b455cdf935bf438c P b5999f170a954efe8fd8a2c9cb624dac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5999f170a954efe8fd8a2c9cb624dac" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 42347 } } peers { permanent_uuid: "f00b23359dc141bd9b6e3794019c4d4d" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 32937 } } peers { permanent_uuid: "99ced08fd0bb4678b13a4dde54b67b03" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 36791 } }
I20260809 15:07:43.010160 32597 heartbeater.cc:503] Master 127.24.183.254:42993 was elected leader, sending a full tablet report...
I20260809 15:07:43.010560 612 raft_consensus.cc:482] T c2d3b40ce1944fd7b455cdf935bf438c P b5999f170a954efe8fd8a2c9cb624dac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:07:43.010854 612 raft_consensus.cc:504] T c2d3b40ce1944fd7b455cdf935bf438c P b5999f170a954efe8fd8a2c9cb624dac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5999f170a954efe8fd8a2c9cb624dac" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 42347 } } peers { permanent_uuid: "f00b23359dc141bd9b6e3794019c4d4d" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 32937 } } peers { permanent_uuid: "99ced08fd0bb4678b13a4dde54b67b03" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 36791 } }
I20260809 15:07:43.011333 599 ts_tablet_manager.cc:1428] T c2d3b40ce1944fd7b455cdf935bf438c P b5999f170a954efe8fd8a2c9cb624dac: Time spent starting tablet: real 0.013s user 0.005s sys 0.009s
I20260809 15:07:43.012704 612 leader_election.cc:288] T c2d3b40ce1944fd7b455cdf935bf438c P b5999f170a954efe8fd8a2c9cb624dac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f00b23359dc141bd9b6e3794019c4d4d (127.24.183.195:32937), 99ced08fd0bb4678b13a4dde54b67b03 (127.24.183.194:36791)
I20260809 15:07:43.016053 592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:43.013050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:51369 (local address 127.24.183.195:32937)
0809 15:07:43.013225 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:43.013232 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:43.013781 (+ 549us) server_negotiation.cc:407] Connection header received
0809 15:07:43.014023 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:43.014028 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:43.014086 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:43.014152 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:43.015310 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:43.015315 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:43.015320 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:07:43.015350 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:43.015388 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:07:43.015404 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:43.015498 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:43.015750 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 15:07:43.015810 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 15:07:43.016268 595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:43.013850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:56799 (local address 127.24.183.194:36791)
0809 15:07:43.014024 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:43.014029 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:43.014567 (+ 538us) server_negotiation.cc:407] Connection header received
0809 15:07:43.014676 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:43.014681 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:43.014734 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:43.014819 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:43.015654 (+ 835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:43.015659 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:43.015662 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:07:43.015719 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:43.015756 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:07:43.015769 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:43.015872 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:43.015969 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 15:07:43.016032 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 15:07:43.016486 615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:43.013764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:36791 (local address 127.24.183.193:56799)
0809 15:07:43.014464 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:43.014482 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:43.014576 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:43.014874 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:43.014878 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:43.014900 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:43.015473 (+ 573us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:07:43.015476 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:43.015489 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:07:43.015524 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:43.015871 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:43.015875 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:43.015880 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 15:07:43.016013 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 15:07:43.016060 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":438,"threads_started":1}
I20260809 15:07:43.016839 614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:43.013079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:32937 (local address 127.24.183.193:51369)
0809 15:07:43.013677 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:43.013697 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:43.013874 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:43.014380 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:43.014385 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:43.014410 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:43.015087 (+ 677us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:07:43.015092 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:43.015106 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:07:43.015150 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:43.015587 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:43.015591 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:43.015595 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 15:07:43.016312 (+ 717us) client_negotiation.cc:240] Negotiation successful
0809 15:07:43.016375 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":308,"threads_started":1}
I20260809 15:07:43.017275 32736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2d3b40ce1944fd7b455cdf935bf438c" candidate_uuid: "b5999f170a954efe8fd8a2c9cb624dac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99ced08fd0bb4678b13a4dde54b67b03" is_pre_election: true
I20260809 15:07:43.017621 32736 raft_consensus.cc:2454] T c2d3b40ce1944fd7b455cdf935bf438c P 99ced08fd0bb4678b13a4dde54b67b03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5999f170a954efe8fd8a2c9cb624dac in term 0.
I20260809 15:07:43.017657 489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2d3b40ce1944fd7b455cdf935bf438c" candidate_uuid: "b5999f170a954efe8fd8a2c9cb624dac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00b23359dc141bd9b6e3794019c4d4d" is_pre_election: true
I20260809 15:07:43.018098 32432 leader_election.cc:302] T c2d3b40ce1944fd7b455cdf935bf438c P b5999f170a954efe8fd8a2c9cb624dac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99ced08fd0bb4678b13a4dde54b67b03, b5999f170a954efe8fd8a2c9cb624dac; no voters:
I20260809 15:07:43.018097 489 raft_consensus.cc:2454] T c2d3b40ce1944fd7b455cdf935bf438c P f00b23359dc141bd9b6e3794019c4d4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5999f170a954efe8fd8a2c9cb624dac in term 0.
I20260809 15:07:43.018417 612 raft_consensus.cc:2781] T c2d3b40ce1944fd7b455cdf935bf438c P b5999f170a954efe8fd8a2c9cb624dac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:07:43.018558 612 raft_consensus.cc:482] T c2d3b40ce1944fd7b455cdf935bf438c P b5999f170a954efe8fd8a2c9cb624dac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:07:43.018679 612 raft_consensus.cc:3037] T c2d3b40ce1944fd7b455cdf935bf438c P b5999f170a954efe8fd8a2c9cb624dac [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:43.020926 612 raft_consensus.cc:504] T c2d3b40ce1944fd7b455cdf935bf438c P b5999f170a954efe8fd8a2c9cb624dac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5999f170a954efe8fd8a2c9cb624dac" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 42347 } } peers { permanent_uuid: "f00b23359dc141bd9b6e3794019c4d4d" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 32937 } } peers { permanent_uuid: "99ced08fd0bb4678b13a4dde54b67b03" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 36791 } }
I20260809 15:07:43.021446 612 leader_election.cc:288] T c2d3b40ce1944fd7b455cdf935bf438c P b5999f170a954efe8fd8a2c9cb624dac [CANDIDATE]: Term 1 election: Requested vote from peers f00b23359dc141bd9b6e3794019c4d4d (127.24.183.195:32937), 99ced08fd0bb4678b13a4dde54b67b03 (127.24.183.194:36791)
I20260809 15:07:43.021797 489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2d3b40ce1944fd7b455cdf935bf438c" candidate_uuid: "b5999f170a954efe8fd8a2c9cb624dac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00b23359dc141bd9b6e3794019c4d4d"
I20260809 15:07:43.021908 32736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2d3b40ce1944fd7b455cdf935bf438c" candidate_uuid: "b5999f170a954efe8fd8a2c9cb624dac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99ced08fd0bb4678b13a4dde54b67b03"
I20260809 15:07:43.021963 489 raft_consensus.cc:3037] T c2d3b40ce1944fd7b455cdf935bf438c P f00b23359dc141bd9b6e3794019c4d4d [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:43.022089 32736 raft_consensus.cc:3037] T c2d3b40ce1944fd7b455cdf935bf438c P 99ced08fd0bb4678b13a4dde54b67b03 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:43.024338 489 raft_consensus.cc:2454] T c2d3b40ce1944fd7b455cdf935bf438c P f00b23359dc141bd9b6e3794019c4d4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5999f170a954efe8fd8a2c9cb624dac in term 1.
I20260809 15:07:43.024361 32736 raft_consensus.cc:2454] T c2d3b40ce1944fd7b455cdf935bf438c P 99ced08fd0bb4678b13a4dde54b67b03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5999f170a954efe8fd8a2c9cb624dac in term 1.
I20260809 15:07:43.024657 32432 leader_election.cc:302] T c2d3b40ce1944fd7b455cdf935bf438c P b5999f170a954efe8fd8a2c9cb624dac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5999f170a954efe8fd8a2c9cb624dac, f00b23359dc141bd9b6e3794019c4d4d; no voters:
I20260809 15:07:43.024900 612 raft_consensus.cc:2781] T c2d3b40ce1944fd7b455cdf935bf438c P b5999f170a954efe8fd8a2c9cb624dac [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:07:43.025043 612 raft_consensus.cc:686] T c2d3b40ce1944fd7b455cdf935bf438c P b5999f170a954efe8fd8a2c9cb624dac [term 1 LEADER]: Becoming Leader. State: Replica: b5999f170a954efe8fd8a2c9cb624dac, State: Running, Role: LEADER
I20260809 15:07:43.025434 612 consensus_queue.cc:237] T c2d3b40ce1944fd7b455cdf935bf438c P b5999f170a954efe8fd8a2c9cb624dac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5999f170a954efe8fd8a2c9cb624dac" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 42347 } } peers { permanent_uuid: "f00b23359dc141bd9b6e3794019c4d4d" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 32937 } } peers { permanent_uuid: "99ced08fd0bb4678b13a4dde54b67b03" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 36791 } }
I20260809 15:07:43.029688 32312 catalog_manager.cc:5495] T c2d3b40ce1944fd7b455cdf935bf438c P b5999f170a954efe8fd8a2c9cb624dac reported cstate change: term changed from 0 to 1, leader changed from <none> to b5999f170a954efe8fd8a2c9cb624dac (127.24.183.193). New cstate: current_term: 1 leader_uuid: "b5999f170a954efe8fd8a2c9cb624dac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5999f170a954efe8fd8a2c9cb624dac" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 42347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f00b23359dc141bd9b6e3794019c4d4d" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 32937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99ced08fd0bb4678b13a4dde54b67b03" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 36791 } health_report { overall_health: UNKNOWN } } }
I20260809 15:07:43.052349 588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:43.050387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56228 (local address 127.24.183.193:42347)
0809 15:07:43.050566 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:43.050573 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:43.050700 (+ 127us) server_negotiation.cc:407] Connection header received
0809 15:07:43.050902 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:43.050907 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:43.051010 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:43.051116 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:43.051861 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:43.051866 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:43.051869 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:07:43.051883 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:43.051916 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:07:43.051928 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:43.052013 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:43.052086 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 15:07:43.052131 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260809 15:07:43.058053 32600 tablet.cc:2369] T c2d3b40ce1944fd7b455cdf935bf438c P b5999f170a954efe8fd8a2c9cb624dac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:07:43.061743 489 raft_consensus.cc:1261] T c2d3b40ce1944fd7b455cdf935bf438c P f00b23359dc141bd9b6e3794019c4d4d [term 1 FOLLOWER]: Refusing update from remote peer b5999f170a954efe8fd8a2c9cb624dac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:07:43.061815 32736 raft_consensus.cc:1261] T c2d3b40ce1944fd7b455cdf935bf438c P 99ced08fd0bb4678b13a4dde54b67b03 [term 1 FOLLOWER]: Refusing update from remote peer b5999f170a954efe8fd8a2c9cb624dac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:07:43.062633 612 consensus_queue.cc:1035] T c2d3b40ce1944fd7b455cdf935bf438c P b5999f170a954efe8fd8a2c9cb624dac [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00b23359dc141bd9b6e3794019c4d4d" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 32937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:07:43.062978 611 consensus_queue.cc:1035] T c2d3b40ce1944fd7b455cdf935bf438c P b5999f170a954efe8fd8a2c9cb624dac [LEADER]: Connected to new peer: Peer: permanent_uuid: "99ced08fd0bb4678b13a4dde54b67b03" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:07:43.074606 619 mvcc.cc:205] Tried to move back new op lower bound from 7316635906289983488 to 7316635906155364352. Current Snapshot: MvccSnapshot[applied={T|T < 7316635906289983488}]
I20260809 15:07:43.088953 32311 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51488:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:07:43.089236 32311 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51488:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:07:43.092967 32311 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 179db7c38112497ca6091221cfd3886f: Sending DeleteTablet for 3 replicas of tablet c2d3b40ce1944fd7b455cdf935bf438c
I20260809 15:07:43.094059 32501 tablet_service.cc:1514] Processing DeleteTablet for tablet c2d3b40ce1944fd7b455cdf935bf438c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:07:43 UTC) from {username='slave'} at 127.0.0.1:56224
I20260809 15:07:43.094414 456 tablet_service.cc:1514] Processing DeleteTablet for tablet c2d3b40ce1944fd7b455cdf935bf438c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:07:43 UTC) from {username='slave'} at 127.0.0.1:57190
I20260809 15:07:43.094576 32704 tablet_service.cc:1514] Processing DeleteTablet for tablet c2d3b40ce1944fd7b455cdf935bf438c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:07:43 UTC) from {username='slave'} at 127.0.0.1:34148
I20260809 15:07:43.095345 632 tablet_replica.cc:330] T c2d3b40ce1944fd7b455cdf935bf438c P f00b23359dc141bd9b6e3794019c4d4d: stopping tablet replica
I20260809 15:07:43.095913 632 raft_consensus.cc:2229] T c2d3b40ce1944fd7b455cdf935bf438c P f00b23359dc141bd9b6e3794019c4d4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:07:43.096438 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 32404
I20260809 15:07:43.096599 632 raft_consensus.cc:2258] T c2d3b40ce1944fd7b455cdf935bf438c P f00b23359dc141bd9b6e3794019c4d4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:07:43.101686 632 ts_tablet_manager.cc:1905] T c2d3b40ce1944fd7b455cdf935bf438c P f00b23359dc141bd9b6e3794019c4d4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:07:43.103452 633 tablet_replica.cc:330] T c2d3b40ce1944fd7b455cdf935bf438c P 99ced08fd0bb4678b13a4dde54b67b03: stopping tablet replica
I20260809 15:07:43.103924 633 raft_consensus.cc:2229] T c2d3b40ce1944fd7b455cdf935bf438c P 99ced08fd0bb4678b13a4dde54b67b03 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:07:43.104447 633 raft_consensus.cc:2258] T c2d3b40ce1944fd7b455cdf935bf438c P 99ced08fd0bb4678b13a4dde54b67b03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:07:43.104791 631 tablet_replica.cc:330] T c2d3b40ce1944fd7b455cdf935bf438c P b5999f170a954efe8fd8a2c9cb624dac: stopping tablet replica
I20260809 15:07:43.105198 631 raft_consensus.cc:2229] T c2d3b40ce1944fd7b455cdf935bf438c P b5999f170a954efe8fd8a2c9cb624dac [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:07:43.106824 633 ts_tablet_manager.cc:1905] T c2d3b40ce1944fd7b455cdf935bf438c P 99ced08fd0bb4678b13a4dde54b67b03: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:07:43.107033 588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:43.099266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56234 (local address 127.24.183.193:42347)
0809 15:07:43.099730 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:43.099736 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:43.103130 (+ 3394us) server_negotiation.cc:407] Connection header received
0809 15:07:43.103555 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:43.103561 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:43.103625 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:43.103733 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:43.106162 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:43.106168 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:43.106172 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:07:43.106192 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:43.106231 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:07:43.106246 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:43.106344 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:43.106619 (+ 275us) server_negotiation.cc:299] Negotiation successful
0809 15:07:43.106681 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316}
I20260809 15:07:43.107492 631 raft_consensus.cc:2258] T c2d3b40ce1944fd7b455cdf935bf438c P b5999f170a954efe8fd8a2c9cb624dac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:07:43.109738 632 ts_tablet_manager.cc:1918] T c2d3b40ce1944fd7b455cdf935bf438c P f00b23359dc141bd9b6e3794019c4d4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:07:43.107995 32591 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:07:43.109917 632 log.cc:1196] T c2d3b40ce1944fd7b455cdf935bf438c P f00b23359dc141bd9b6e3794019c4d4d: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-2/wal/wals/c2d3b40ce1944fd7b455cdf935bf438c
I20260809 15:07:43.110455 632 ts_tablet_manager.cc:1939] T c2d3b40ce1944fd7b455cdf935bf438c P f00b23359dc141bd9b6e3794019c4d4d: Deleting consensus metadata
I20260809 15:07:43.112804 32290 catalog_manager.cc:4841] TS f00b23359dc141bd9b6e3794019c4d4d (127.24.183.195:32937): tablet c2d3b40ce1944fd7b455cdf935bf438c (table test-table [id=40c5c61fbe2f4643a112a93ad4d7a0f0]) successfully deleted
I20260809 15:07:43.113502 633 ts_tablet_manager.cc:1918] T c2d3b40ce1944fd7b455cdf935bf438c P 99ced08fd0bb4678b13a4dde54b67b03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:07:43.113688 633 log.cc:1196] T c2d3b40ce1944fd7b455cdf935bf438c P 99ced08fd0bb4678b13a4dde54b67b03: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288021313853-25311-0/minicluster-data/ts-1/wal/wals/c2d3b40ce1944fd7b455cdf935bf438c
I20260809 15:07:43.114277 633 ts_tablet_manager.cc:1939] T c2d3b40ce1944fd7b455cdf935bf438c P 99ced08fd0bb4678b13a4dde54b67b03: Deleting consensus metadata
I20260809 15:07:43.116212 32290 catalog_manager.cc:4841] TS 99ced08fd0bb4678b13a4dde54b67b03 (127.24.183.194:36791): tablet c2d3b40ce1944fd7b455cdf935bf438c (table test-table [id=40c5c61fbe2f4643a112a93ad4d7a0f0]) successfully deleted
I20260809 15:07:43.207420 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 32404
I20260809 15:07:43.228937 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 32604
W20260809 15:07:43.228942 32290 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.183.193:42347 (error 108)
I20260809 15:07:43.230990 594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:43.230339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:42347 (local address 127.0.0.1:56242)
0809 15:07:43.230531 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:43.230718 (+ 187us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:42347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 15:07:43.231205 32290 catalog_manager.cc:4584] TS b5999f170a954efe8fd8a2c9cb624dac (127.24.183.193:42347): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c2d3b40ce1944fd7b455cdf935bf438c: Network error: Client connection negotiation failed: client connection to 127.24.183.193:42347: connect: Connection refused (error 111)
I20260809 15:07:43.231957 595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:43.229755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34156 (local address 127.24.183.194:36791)
0809 15:07:43.229928 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:43.229934 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:43.229998 (+ 64us) server_negotiation.cc:407] Connection header received
0809 15:07:43.230210 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:43.230214 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:43.230311 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:43.230401 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:43.231309 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:43.231316 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:43.231320 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:07:43.231339 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:43.231376 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:07:43.231392 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:43.231517 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:43.231590 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 15:07:43.231655 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 15:07:43.232610 334 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:07:43.284699 594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:43.284172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:42347 (local address 127.0.0.1:56250)
0809 15:07:43.284394 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:43.284520 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:42347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 15:07:43.285753 594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:43.285328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:42347 (local address 127.0.0.1:56254)
0809 15:07:43.285488 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:43.285574 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:42347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 15:07:43.323380 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 32604
I20260809 15:07:43.346369 594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:43.345786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:42347 (local address 127.0.0.1:56260)
0809 15:07:43.346042 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:43.346167 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:42347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260809 15:07:43.347481 594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:43.347075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:42347 (local address 127.0.0.1:56270)
0809 15:07:43.347229 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:43.347310 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:42347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 15:07:43.347528 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 347
I20260809 15:07:43.350275 592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:43.348281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57198 (local address 127.24.183.195:32937)
0809 15:07:43.348470 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:43.348475 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:43.348495 (+ 20us) server_negotiation.cc:407] Connection header received
0809 15:07:43.348604 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:43.348610 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:43.348712 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:43.348803 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:43.349636 (+ 833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:43.349642 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:43.349647 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:07:43.349664 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:43.349696 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:07:43.349708 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:43.349821 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:43.349938 (+ 117us) server_negotiation.cc:299] Negotiation successful
0809 15:07:43.349996 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 15:07:43.350884 560 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:07:43.428086 594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:43.427601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:42347 (local address 127.0.0.1:56284)
0809 15:07:43.427830 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:43.427938 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:42347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260809 15:07:43.429212 594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:43.428803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:42347 (local address 127.0.0.1:56300)
0809 15:07:43.428999 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:43.429072 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:42347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260809 15:07:43.439080 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 347
I20260809 15:07:43.463079 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 32267
I20260809 15:07:43.464306 32355 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:07:43.571756 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 32267
2026-08-09T15:07:43Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2526 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 15:07:43.619802 25311 mini_oidc.cc:152] Starting JWKS server
I20260809 15:07:43.622678 25311 webserver.cc:460] Webserver started at https://127.0.0.1:44769/ using document root <none> and password file <none>
I20260809 15:07:43.623049 25311 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 15:07:43.623656 25311 webserver.cc:460] Webserver started at http://127.0.0.1:39163/ using document root <none> and password file <none>
2026-08-09T15:07:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:07:43Z Disabled control of system clock
I20260809 15:07:43.655213 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:40385
--webserver_interface=127.24.183.254
--webserver_port=0
--builtin_ntp_servers=127.24.183.212:40737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:40385
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44769/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288021313853-25311-0/testchainca.cert with env {}
W20260809 15:07:43.977679 651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:43.977972 651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:43.978103 651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:43.985097 651 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:07:43.985203 651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:43.985270 651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:43.985347 651 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:07:43.985406 651 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 15:07:43.985487 651 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 15:07:43.985538 651 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44769/jwks.json
I20260809 15:07:43.995014 651 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:40737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:40385
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:40385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44769/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288021313853-25311-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:43.996302 651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:43.997519 651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:44.008582 657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:44.008632 659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:44.008658 661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:44.009619 651 server_base.cc:1034] running on GCE node
I20260809 15:07:44.010370 651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:44.011641 651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:44.012879 651 hybrid_clock.cc:648] HybridClock initialized: now 1786288064012842 us; error 34 us; skew 500 ppm
I20260809 15:07:44.013195 651 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:07:44.015411 651 webserver.cc:460] Webserver started at http://127.24.183.254:44549/ using document root <none> and password file <none>
I20260809 15:07:44.016098 651 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:44.016202 651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:44.016503 651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:44.019287 651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "3d7b94991c424efe8940b936a633d4a8"
format_stamp: "Formatted at 2026-08-09 15:07:44 on dist-test-slave-nj21"
server_key: "9209c7dc13efb2d272718fce575aadd6"
server_key_iv: "f79ae558a4a5b2dd32eb2c750445737a"
server_key_version: "encryptionkey@0"
I20260809 15:07:44.020074 651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "3d7b94991c424efe8940b936a633d4a8"
format_stamp: "Formatted at 2026-08-09 15:07:44 on dist-test-slave-nj21"
server_key: "9209c7dc13efb2d272718fce575aadd6"
server_key_iv: "f79ae558a4a5b2dd32eb2c750445737a"
server_key_version: "encryptionkey@0"
I20260809 15:07:44.024816 651 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 15:07:44.028026 668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:44.029290 651 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 15:07:44.029474 651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "3d7b94991c424efe8940b936a633d4a8"
format_stamp: "Formatted at 2026-08-09 15:07:44 on dist-test-slave-nj21"
server_key: "9209c7dc13efb2d272718fce575aadd6"
server_key_iv: "f79ae558a4a5b2dd32eb2c750445737a"
server_key_version: "encryptionkey@0"
I20260809 15:07:44.029672 651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:44.063406 651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:44.064252 651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:44.064543 651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:44.081789 651 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:40385
I20260809 15:07:44.081792 756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:40385 every 8 connection(s)
I20260809 15:07:44.083079 651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:07:44.086663 757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:44.093109 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 651
I20260809 15:07:44.093343 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:07:44.094195 25311 external_mini_cluster.cc:1428] Setting key b823edf639c598f8585ba5e47d7087fc
I20260809 15:07:44.097898 757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d7b94991c424efe8940b936a633d4a8: Bootstrap starting.
I20260809 15:07:44.100951 757 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d7b94991c424efe8940b936a633d4a8: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:44.101959 757 log.cc:824] T 00000000000000000000000000000000 P 3d7b94991c424efe8940b936a633d4a8: Log is configured to *not* fsync() on all Append() calls
I20260809 15:07:44.104660 757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d7b94991c424efe8940b936a633d4a8: No bootstrap required, opened a new log
I20260809 15:07:44.107522 760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:44.096187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37152 (local address 127.24.183.254:40385)
0809 15:07:44.097104 (+ 917us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:44.097114 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:44.097156 (+ 42us) server_negotiation.cc:407] Connection header received
0809 15:07:44.098880 (+ 1724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:44.098904 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:44.099346 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:44.099716 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:44.100355 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:44.101647 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:44.103297 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:44.103965 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:44.104128 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:44.105276 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:44.105284 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:44.105297 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:07:44.105801 (+ 504us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:44.105882 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:07:44.105905 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:44.106006 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:44.106234 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 15:07:44.106521 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":320,"threads_started":1}
I20260809 15:07:44.111117 25311 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:07:44.112402 757 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d7b94991c424efe8940b936a633d4a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7b94991c424efe8940b936a633d4a8" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 40385 } }
I20260809 15:07:44.112715 757 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d7b94991c424efe8940b936a633d4a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:44.112810 757 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d7b94991c424efe8940b936a633d4a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d7b94991c424efe8940b936a633d4a8, State: Initialized, Role: FOLLOWER
I20260809 15:07:44.113423 757 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d7b94991c424efe8940b936a633d4a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7b94991c424efe8940b936a633d4a8" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 40385 } }
I20260809 15:07:44.113649 757 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d7b94991c424efe8940b936a633d4a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:07:44.113785 757 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d7b94991c424efe8940b936a633d4a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:07:44.113952 757 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d7b94991c424efe8940b936a633d4a8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:44.116266 757 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d7b94991c424efe8940b936a633d4a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7b94991c424efe8940b936a633d4a8" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 40385 } }
I20260809 15:07:44.116724 757 leader_election.cc:302] T 00000000000000000000000000000000 P 3d7b94991c424efe8940b936a633d4a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d7b94991c424efe8940b936a633d4a8; no voters:
I20260809 15:07:44.117257 757 leader_election.cc:288] T 00000000000000000000000000000000 P 3d7b94991c424efe8940b936a633d4a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:07:44.117388 762 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d7b94991c424efe8940b936a633d4a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:07:44.118057 762 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d7b94991c424efe8940b936a633d4a8 [term 1 LEADER]: Becoming Leader. State: Replica: 3d7b94991c424efe8940b936a633d4a8, State: Running, Role: LEADER
I20260809 15:07:44.118526 762 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d7b94991c424efe8940b936a633d4a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7b94991c424efe8940b936a633d4a8" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 40385 } }
I20260809 15:07:44.119208 757 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d7b94991c424efe8940b936a633d4a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:07:44.123509 765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d7b94991c424efe8940b936a633d4a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d7b94991c424efe8940b936a633d4a8. Latest consensus state: current_term: 1 leader_uuid: "3d7b94991c424efe8940b936a633d4a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7b94991c424efe8940b936a633d4a8" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 40385 } } }
I20260809 15:07:44.123533 764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d7b94991c424efe8940b936a633d4a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d7b94991c424efe8940b936a633d4a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7b94991c424efe8940b936a633d4a8" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 40385 } } }
I20260809 15:07:44.124051 765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d7b94991c424efe8940b936a633d4a8 [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:44.124048 764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d7b94991c424efe8940b936a633d4a8 [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:44.125406 778 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:07:44.130406 778 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:07:44.139056 778 catalog_manager.cc:1347] Generated new cluster ID: 762a02f3793d402db18832ee02d278e7
I20260809 15:07:44.139178 778 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:07:44.154728 778 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:07:44.156054 778 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:07:44.163470 778 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d7b94991c424efe8940b936a633d4a8: Generated new TSK 0
I20260809 15:07:44.164250 778 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 15:07:44.207142 794 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 15:07:44.207456 794 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 15:07:44.208405 760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:44.187343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37156 (local address 127.24.183.254:40385)
0809 15:07:44.187562 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:44.187569 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:44.188035 (+ 466us) server_negotiation.cc:407] Connection header received
0809 15:07:44.188382 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:44.188387 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:44.188463 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:44.188578 (+ 115us) server_negotiation.cc:226] Negotiated authn=JWT
0809 15:07:44.189193 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:44.189944 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:44.192038 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:44.192464 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:44.192553 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:44.193128 (+ 575us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 15:07:44.207928 (+ 14800us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44769/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 15:07:44.208191 (+ 263us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37156: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44769/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":71}
W20260809 15:07:44.208509 760 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37156: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44769/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260809 15:07:44.210330 793 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:07:44.187245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:40385 (local address 127.0.0.1:37156)
0809 15:07:44.187935 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:44.187956 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:44.188133 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:44.188617 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:44.188623 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:44.188637 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0809 15:07:44.189051 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:44.189063 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:44.190080 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:44.190086 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:44.191902 (+ 1816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:44.191917 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:44.192629 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:44.192636 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:44.192818 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:44.192895 (+ 77us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 15:07:44.208201 (+ 15306us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44769/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 15:07:44.208542 (+ 341us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.183.254:40385: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44769/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":305,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.24.183.254:40385: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44769/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 15:07:44.212206 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 651
I20260809 15:07:44.213291 751 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:07:44.316239 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 651
2026-08-09T15:07:44Z chronyd exiting
I20260809 15:07:44.374876 25311 test_util.cc:175] -----------------------------------------------
I20260809 15:07:44.375015 25311 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288021313853-25311-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (760 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 15:07:44.378782 25311 mini_oidc.cc:152] Starting JWKS server
I20260809 15:07:44.380621 25311 webserver.cc:460] Webserver started at https://127.0.0.1:46785/ using document root <none> and password file <none>
I20260809 15:07:44.380838 25311 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 15:07:44.381453 25311 webserver.cc:460] Webserver started at http://127.0.0.1:37281/ using document root <none> and password file <none>
2026-08-09T15:07:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:07:44Z Disabled control of system clock
I20260809 15:07:44.411334 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:43183
--webserver_interface=127.24.183.254
--webserver_port=0
--builtin_ntp_servers=127.24.183.212:33267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:43183
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46785/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288021313853-25311-0/testchainca.cert with env {}
W20260809 15:07:44.731901 803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:44.732193 803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:44.732326 803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:44.739398 803 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:07:44.739511 803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:44.739576 803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:44.739657 803 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:07:44.739745 803 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 15:07:44.739833 803 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 15:07:44.739888 803 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46785/jwks.json
I20260809 15:07:44.749756 803 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:33267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:43183
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:43183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46785/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288021313853-25311-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:44.751032 803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:44.752324 803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:44.763481 817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:44.763573 814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:44.763600 813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:44.764430 803 server_base.cc:1034] running on GCE node
I20260809 15:07:44.765098 803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:44.766394 803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:44.767583 803 hybrid_clock.cc:648] HybridClock initialized: now 1786288064767543 us; error 28 us; skew 500 ppm
I20260809 15:07:44.767925 803 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:07:44.770078 803 webserver.cc:460] Webserver started at http://127.24.183.254:37595/ using document root <none> and password file <none>
I20260809 15:07:44.770714 803 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:44.770815 803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:44.771133 803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:44.773937 803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "ee526100dcd9445ea95761fdf3e7e15a"
format_stamp: "Formatted at 2026-08-09 15:07:44 on dist-test-slave-nj21"
server_key: "04a661bc444e1027be85b4367d6b1002"
server_key_iv: "c518eafd3037b53b79fe732c542f3912"
server_key_version: "encryptionkey@0"
I20260809 15:07:44.774693 803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "ee526100dcd9445ea95761fdf3e7e15a"
format_stamp: "Formatted at 2026-08-09 15:07:44 on dist-test-slave-nj21"
server_key: "04a661bc444e1027be85b4367d6b1002"
server_key_iv: "c518eafd3037b53b79fe732c542f3912"
server_key_version: "encryptionkey@0"
I20260809 15:07:44.779497 803 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260809 15:07:44.782688 832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:44.783921 803 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:07:44.784083 803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "ee526100dcd9445ea95761fdf3e7e15a"
format_stamp: "Formatted at 2026-08-09 15:07:44 on dist-test-slave-nj21"
server_key: "04a661bc444e1027be85b4367d6b1002"
server_key_iv: "c518eafd3037b53b79fe732c542f3912"
server_key_version: "encryptionkey@0"
I20260809 15:07:44.784271 803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:44.806278 803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:44.807044 803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:44.807324 803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:44.824493 803 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:43183
I20260809 15:07:44.824501 917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:43183 every 8 connection(s)
I20260809 15:07:44.825826 803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:07:44.827198 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 803
I20260809 15:07:44.827409 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:07:44.828279 25311 external_mini_cluster.cc:1428] Setting key 2e8c4b966e643a0d94af9e1c57413a28
I20260809 15:07:44.830454 918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:44.842808 925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:44.830137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40464 (local address 127.24.183.254:43183)
0809 15:07:44.831141 (+ 1004us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:44.831154 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:44.831205 (+ 51us) server_negotiation.cc:407] Connection header received
0809 15:07:44.833324 (+ 2119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:44.833360 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:44.833917 (+ 557us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:44.834344 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:44.834837 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:44.836818 (+ 1981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:44.838035 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:44.839017 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:44.839192 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:44.840195 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:44.840206 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:44.840222 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:07:44.840899 (+ 677us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:44.841008 (+ 109us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:07:44.841033 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:44.841151 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:44.841282 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 15:07:44.841597 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":326,"threads_started":1}
I20260809 15:07:44.844368 918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee526100dcd9445ea95761fdf3e7e15a: Bootstrap starting.
I20260809 15:07:44.846561 25311 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:07:44.847970 918 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee526100dcd9445ea95761fdf3e7e15a: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:44.849133 918 log.cc:824] T 00000000000000000000000000000000 P ee526100dcd9445ea95761fdf3e7e15a: Log is configured to *not* fsync() on all Append() calls
I20260809 15:07:44.851745 918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee526100dcd9445ea95761fdf3e7e15a: No bootstrap required, opened a new log
I20260809 15:07:44.858809 918 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee526100dcd9445ea95761fdf3e7e15a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee526100dcd9445ea95761fdf3e7e15a" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 43183 } }
I20260809 15:07:44.859129 918 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee526100dcd9445ea95761fdf3e7e15a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:44.859225 918 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee526100dcd9445ea95761fdf3e7e15a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee526100dcd9445ea95761fdf3e7e15a, State: Initialized, Role: FOLLOWER
I20260809 15:07:44.859843 918 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee526100dcd9445ea95761fdf3e7e15a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee526100dcd9445ea95761fdf3e7e15a" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 43183 } }
I20260809 15:07:44.860054 918 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee526100dcd9445ea95761fdf3e7e15a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:07:44.860193 918 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee526100dcd9445ea95761fdf3e7e15a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:07:44.860342 918 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee526100dcd9445ea95761fdf3e7e15a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:44.862352 918 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee526100dcd9445ea95761fdf3e7e15a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee526100dcd9445ea95761fdf3e7e15a" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 43183 } }
I20260809 15:07:44.862807 918 leader_election.cc:302] T 00000000000000000000000000000000 P ee526100dcd9445ea95761fdf3e7e15a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee526100dcd9445ea95761fdf3e7e15a; no voters:
I20260809 15:07:44.863391 918 leader_election.cc:288] T 00000000000000000000000000000000 P ee526100dcd9445ea95761fdf3e7e15a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:07:44.863487 928 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee526100dcd9445ea95761fdf3e7e15a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:07:44.864171 928 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee526100dcd9445ea95761fdf3e7e15a [term 1 LEADER]: Becoming Leader. State: Replica: ee526100dcd9445ea95761fdf3e7e15a, State: Running, Role: LEADER
I20260809 15:07:44.864698 928 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee526100dcd9445ea95761fdf3e7e15a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee526100dcd9445ea95761fdf3e7e15a" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 43183 } }
I20260809 15:07:44.865294 918 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee526100dcd9445ea95761fdf3e7e15a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:07:44.867654 932 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee526100dcd9445ea95761fdf3e7e15a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee526100dcd9445ea95761fdf3e7e15a. Latest consensus state: current_term: 1 leader_uuid: "ee526100dcd9445ea95761fdf3e7e15a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee526100dcd9445ea95761fdf3e7e15a" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 43183 } } }
I20260809 15:07:44.867745 930 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee526100dcd9445ea95761fdf3e7e15a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee526100dcd9445ea95761fdf3e7e15a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee526100dcd9445ea95761fdf3e7e15a" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 43183 } } }
I20260809 15:07:44.868072 932 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee526100dcd9445ea95761fdf3e7e15a [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:44.868121 930 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee526100dcd9445ea95761fdf3e7e15a [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:44.869295 940 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:07:44.875341 940 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:07:44.883762 940 catalog_manager.cc:1347] Generated new cluster ID: daf80ba872ef44368b0f02266da893f3
I20260809 15:07:44.883881 940 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:07:44.905850 940 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:07:44.907642 940 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:07:44.920372 940 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee526100dcd9445ea95761fdf3e7e15a: Generated new TSK 0
I20260809 15:07:44.921361 940 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 15:07:44.959214 966 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 15:07:44.959347 966 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 15:07:44.960415 925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:44.934091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40472 (local address 127.24.183.254:43183)
0809 15:07:44.934272 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:44.934278 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:44.934615 (+ 337us) server_negotiation.cc:407] Connection header received
0809 15:07:44.934835 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:44.934840 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:44.934911 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:44.935033 (+ 122us) server_negotiation.cc:226] Negotiated authn=JWT
0809 15:07:44.935480 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:44.936250 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:44.937592 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:44.938068 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:44.938176 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:44.938646 (+ 470us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 15:07:44.959709 (+ 21063us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46785/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 15:07:44.960068 (+ 359us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40472: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46785/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":53}
W20260809 15:07:44.960552 925 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40472: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46785/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260809 15:07:44.960609 965 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:07:44.933917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:43183 (local address 127.0.0.1:40472)
0809 15:07:44.934518 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:44.934537 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:44.934670 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:44.935062 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:44.935067 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:44.935079 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0809 15:07:44.935324 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:44.935335 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:44.936387 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:44.936393 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:44.937456 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:44.937467 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:44.938224 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:44.938230 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:44.938374 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:44.938407 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 15:07:44.959938 (+ 21531us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46785/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 15:07:44.960194 (+ 256us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.183.254:43183: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46785/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":395,"spinlock_wait_cycles":22912,"thread_start_us":378,"threads_started":1}
I20260809 15:07:44.962064 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 803
I20260809 15:07:44.963224 913 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:07:45.056000 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 803
2026-08-09T15:07:45Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (739 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 15:07:45.117945 25311 mini_oidc.cc:152] Starting JWKS server
I20260809 15:07:45.120494 25311 webserver.cc:460] Webserver started at https://127.0.0.1:33495/ using document root <none> and password file <none>
I20260809 15:07:45.120704 25311 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 15:07:45.121315 25311 webserver.cc:460] Webserver started at http://127.0.0.1:40065/ using document root <none> and password file <none>
2026-08-09T15:07:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:07:45Z Disabled control of system clock
I20260809 15:07:45.152757 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:38253
--webserver_interface=127.24.183.254
--webserver_port=0
--builtin_ntp_servers=127.24.183.212:39975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:38253
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33495/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 15:07:45.473580 973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:45.473836 973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:45.473987 973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:45.480988 973 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:07:45.481081 973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:45.481144 973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:45.481215 973 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:07:45.481274 973 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 15:07:45.481357 973 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 15:07:45.481405 973 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33495/jwks.json
I20260809 15:07:45.490232 973 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:39975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:38253
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:38253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33495/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:45.491452 973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:45.493211 973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:45.504230 979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:45.504290 983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:45.504318 980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:45.505318 973 server_base.cc:1034] running on GCE node
I20260809 15:07:45.506002 973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:45.507267 973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:45.508460 973 hybrid_clock.cc:648] HybridClock initialized: now 1786288065508421 us; error 25 us; skew 500 ppm
I20260809 15:07:45.508792 973 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:07:45.510941 973 webserver.cc:460] Webserver started at http://127.24.183.254:41725/ using document root <none> and password file <none>
I20260809 15:07:45.511579 973 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:45.511689 973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:45.512012 973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:45.514765 973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "87fe9e102d004699adf4728f2a5ccd1c"
format_stamp: "Formatted at 2026-08-09 15:07:45 on dist-test-slave-nj21"
server_key: "bbb03715700b95d42b840dbb47f0d5aa"
server_key_iv: "3d00c27b1d3550e64fb58ccd23734264"
server_key_version: "encryptionkey@0"
I20260809 15:07:45.515511 973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "87fe9e102d004699adf4728f2a5ccd1c"
format_stamp: "Formatted at 2026-08-09 15:07:45 on dist-test-slave-nj21"
server_key: "bbb03715700b95d42b840dbb47f0d5aa"
server_key_iv: "3d00c27b1d3550e64fb58ccd23734264"
server_key_version: "encryptionkey@0"
I20260809 15:07:45.520255 973 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 15:07:45.523401 993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:45.524590 973 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 15:07:45.524735 973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "87fe9e102d004699adf4728f2a5ccd1c"
format_stamp: "Formatted at 2026-08-09 15:07:45 on dist-test-slave-nj21"
server_key: "bbb03715700b95d42b840dbb47f0d5aa"
server_key_iv: "3d00c27b1d3550e64fb58ccd23734264"
server_key_version: "encryptionkey@0"
I20260809 15:07:45.524926 973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:45.547461 973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:45.548298 973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:45.548591 973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:45.565650 973 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:38253
I20260809 15:07:45.565670 1085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:38253 every 8 connection(s)
I20260809 15:07:45.566933 973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:07:45.568958 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 973
I20260809 15:07:45.569197 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:07:45.570060 25311 external_mini_cluster.cc:1428] Setting key 919a1d3f5a21bffe01ae27916ddaff80
I20260809 15:07:45.571260 1086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:45.583024 1089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:45.571945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50776 (local address 127.24.183.254:38253)
0809 15:07:45.572893 (+ 948us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:45.572904 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:45.572946 (+ 42us) server_negotiation.cc:407] Connection header received
0809 15:07:45.574440 (+ 1494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:45.574461 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:45.574882 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:45.575198 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:45.575752 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:45.577084 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:45.578857 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:45.579483 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:45.579614 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:45.580834 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:45.580844 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:45.580857 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:07:45.581385 (+ 528us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:45.581466 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:07:45.581489 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:45.581575 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:45.581808 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 15:07:45.582107 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":366,"threads_started":1}
I20260809 15:07:45.584498 1086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87fe9e102d004699adf4728f2a5ccd1c: Bootstrap starting.
I20260809 15:07:45.586663 25311 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:07:45.587742 1086 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87fe9e102d004699adf4728f2a5ccd1c: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:45.588691 1086 log.cc:824] T 00000000000000000000000000000000 P 87fe9e102d004699adf4728f2a5ccd1c: Log is configured to *not* fsync() on all Append() calls
I20260809 15:07:45.591387 1086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87fe9e102d004699adf4728f2a5ccd1c: No bootstrap required, opened a new log
I20260809 15:07:45.598407 1086 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87fe9e102d004699adf4728f2a5ccd1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87fe9e102d004699adf4728f2a5ccd1c" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 38253 } }
I20260809 15:07:45.598737 1086 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87fe9e102d004699adf4728f2a5ccd1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:45.598827 1086 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87fe9e102d004699adf4728f2a5ccd1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87fe9e102d004699adf4728f2a5ccd1c, State: Initialized, Role: FOLLOWER
I20260809 15:07:45.599320 1086 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87fe9e102d004699adf4728f2a5ccd1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87fe9e102d004699adf4728f2a5ccd1c" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 38253 } }
I20260809 15:07:45.599555 1086 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87fe9e102d004699adf4728f2a5ccd1c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:07:45.599696 1086 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87fe9e102d004699adf4728f2a5ccd1c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:07:45.599861 1086 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87fe9e102d004699adf4728f2a5ccd1c [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:45.601874 1086 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87fe9e102d004699adf4728f2a5ccd1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87fe9e102d004699adf4728f2a5ccd1c" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 38253 } }
I20260809 15:07:45.602337 1086 leader_election.cc:302] T 00000000000000000000000000000000 P 87fe9e102d004699adf4728f2a5ccd1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87fe9e102d004699adf4728f2a5ccd1c; no voters:
I20260809 15:07:45.602905 1086 leader_election.cc:288] T 00000000000000000000000000000000 P 87fe9e102d004699adf4728f2a5ccd1c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:07:45.602990 1091 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87fe9e102d004699adf4728f2a5ccd1c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:07:45.603696 1091 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87fe9e102d004699adf4728f2a5ccd1c [term 1 LEADER]: Becoming Leader. State: Replica: 87fe9e102d004699adf4728f2a5ccd1c, State: Running, Role: LEADER
I20260809 15:07:45.604231 1091 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87fe9e102d004699adf4728f2a5ccd1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87fe9e102d004699adf4728f2a5ccd1c" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 38253 } }
I20260809 15:07:45.604827 1086 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87fe9e102d004699adf4728f2a5ccd1c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:07:45.606854 1092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87fe9e102d004699adf4728f2a5ccd1c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "87fe9e102d004699adf4728f2a5ccd1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87fe9e102d004699adf4728f2a5ccd1c" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 38253 } } }
I20260809 15:07:45.607424 1092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87fe9e102d004699adf4728f2a5ccd1c [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:45.607789 1093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87fe9e102d004699adf4728f2a5ccd1c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87fe9e102d004699adf4728f2a5ccd1c. Latest consensus state: current_term: 1 leader_uuid: "87fe9e102d004699adf4728f2a5ccd1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87fe9e102d004699adf4728f2a5ccd1c" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 38253 } } }
I20260809 15:07:45.608140 1093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87fe9e102d004699adf4728f2a5ccd1c [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:45.609516 1105 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:07:45.614595 1105 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:07:45.622624 1105 catalog_manager.cc:1347] Generated new cluster ID: 73c50649d95041cf97190357a36e6513
I20260809 15:07:45.622716 1105 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:07:45.657644 1105 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:07:45.659567 1105 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:07:45.675626 1105 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87fe9e102d004699adf4728f2a5ccd1c: Generated new TSK 0
I20260809 15:07:45.676754 1105 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:07:45.705320 1089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:45.698135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50782 (local address 127.24.183.254:38253)
0809 15:07:45.698314 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:45.698321 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:45.698786 (+ 465us) server_negotiation.cc:407] Connection header received
0809 15:07:45.699023 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:45.699029 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:45.699120 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:45.699255 (+ 135us) server_negotiation.cc:226] Negotiated authn=JWT
0809 15:07:45.699773 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:45.700481 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:45.701816 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:45.702300 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:45.702414 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:45.702915 (+ 501us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 15:07:45.704717 (+ 1802us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33495/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 15:07:45.705024 (+ 307us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50782: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33495/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":55}
W20260809 15:07:45.705535 1125 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:07:45.698118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:38253 (local address 127.0.0.1:50782)
0809 15:07:45.698692 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:45.698707 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:45.698818 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:45.699297 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:45.699303 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:45.699316 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0809 15:07:45.699601 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:45.699611 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:45.700620 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:45.700624 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:45.701680 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:45.701691 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:45.702489 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:45.702496 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:45.702622 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:45.702653 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 15:07:45.704896 (+ 2243us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33495/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 15:07:45.705138 (+ 242us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.183.254:38253: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33495/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":306,"threads_started":1}
W20260809 15:07:45.705863 1089 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50782: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33495/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260809 15:07:45.706682 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 973
I20260809 15:07:45.707928 1076 generic_service.cc:196] Checking for leaks (request via RPC)
E20260809 15:07:45.722178 1126 webserver.cc:533] Webserver: sslize failed:
E20260809 15:07:45.722302 1126 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 15:07:45.806102 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 973
2026-08-09T15:07:45Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (751 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T15:07:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:07:45Z Disabled control of system clock
I20260809 15:07:45.899295 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:44939
--webserver_interface=127.24.183.254
--webserver_port=0
--builtin_ntp_servers=127.24.183.212:41277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:44939
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288021313853-25311-0/testchainca.cert with env {}
W20260809 15:07:46.222115 1133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:46.222406 1133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:46.222502 1133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:46.229763 1133 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:07:46.229861 1133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:46.229904 1133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:46.229959 1133 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:07:46.230002 1133 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 15:07:46.230062 1133 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 15:07:46.230099 1133 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 15:07:46.239775 1133 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:41277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:44939
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:44939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288021313853-25311-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:46.241019 1133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:46.242281 1133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:46.253639 1139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:46.253662 1141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:46.253919 1143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:46.254406 1133 server_base.cc:1034] running on GCE node
I20260809 15:07:46.255163 1133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:46.256575 1133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:46.257773 1133 hybrid_clock.cc:648] HybridClock initialized: now 1786288066257716 us; error 46 us; skew 500 ppm
I20260809 15:07:46.258091 1133 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:07:46.260329 1133 webserver.cc:460] Webserver started at http://127.24.183.254:35457/ using document root <none> and password file <none>
I20260809 15:07:46.260969 1133 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:46.261057 1133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:46.261327 1133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:46.264233 1133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "4b68eb3436824dce87c3e372d131799f"
format_stamp: "Formatted at 2026-08-09 15:07:46 on dist-test-slave-nj21"
server_key: "ce38e51ae85d2968cb9e20827637c8a6"
server_key_iv: "597787109f4d14d049e5c106380c3228"
server_key_version: "encryptionkey@0"
I20260809 15:07:46.264999 1133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "4b68eb3436824dce87c3e372d131799f"
format_stamp: "Formatted at 2026-08-09 15:07:46 on dist-test-slave-nj21"
server_key: "ce38e51ae85d2968cb9e20827637c8a6"
server_key_iv: "597787109f4d14d049e5c106380c3228"
server_key_version: "encryptionkey@0"
I20260809 15:07:46.269995 1133 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 15:07:46.273280 1152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:46.274528 1133 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 15:07:46.274672 1133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "4b68eb3436824dce87c3e372d131799f"
format_stamp: "Formatted at 2026-08-09 15:07:46 on dist-test-slave-nj21"
server_key: "ce38e51ae85d2968cb9e20827637c8a6"
server_key_iv: "597787109f4d14d049e5c106380c3228"
server_key_version: "encryptionkey@0"
I20260809 15:07:46.274844 1133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:46.289959 1133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:46.290710 1133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:46.290943 1133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:46.308269 1133 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:44939
I20260809 15:07:46.308276 1237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:44939 every 8 connection(s)
I20260809 15:07:46.309559 1133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:07:46.313225 1238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:46.317687 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 1133
I20260809 15:07:46.317898 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:07:46.318728 25311 external_mini_cluster.cc:1428] Setting key e412cf30c2770342e1b40aa85c1de28c
I20260809 15:07:46.325546 1238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b68eb3436824dce87c3e372d131799f: Bootstrap starting.
I20260809 15:07:46.329540 1238 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b68eb3436824dce87c3e372d131799f: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:46.330704 1238 log.cc:824] T 00000000000000000000000000000000 P 4b68eb3436824dce87c3e372d131799f: Log is configured to *not* fsync() on all Append() calls
I20260809 15:07:46.333739 1238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b68eb3436824dce87c3e372d131799f: No bootstrap required, opened a new log
I20260809 15:07:46.336238 1241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:46.320894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34552 (local address 127.24.183.254:44939)
0809 15:07:46.322026 (+ 1132us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:46.322041 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:46.324255 (+ 2214us) server_negotiation.cc:407] Connection header received
0809 15:07:46.326570 (+ 2315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:46.326597 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:46.327154 (+ 557us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:46.327593 (+ 439us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:46.328158 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:46.330093 (+ 1935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:46.331460 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:46.332318 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:46.332493 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:46.333492 (+ 999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:46.333503 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:46.333520 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:07:46.334244 (+ 724us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:46.334331 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:07:46.334357 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:46.334480 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:46.334657 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 15:07:46.334959 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":359,"threads_started":1}
I20260809 15:07:46.339793 25311 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:07:46.343007 1238 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b68eb3436824dce87c3e372d131799f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b68eb3436824dce87c3e372d131799f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 44939 } }
I20260809 15:07:46.343351 1238 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b68eb3436824dce87c3e372d131799f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:46.343456 1238 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b68eb3436824dce87c3e372d131799f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b68eb3436824dce87c3e372d131799f, State: Initialized, Role: FOLLOWER
I20260809 15:07:46.344048 1238 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b68eb3436824dce87c3e372d131799f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b68eb3436824dce87c3e372d131799f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 44939 } }
I20260809 15:07:46.344266 1238 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b68eb3436824dce87c3e372d131799f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:07:46.344403 1238 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b68eb3436824dce87c3e372d131799f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:07:46.344558 1238 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b68eb3436824dce87c3e372d131799f [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:46.346856 1238 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b68eb3436824dce87c3e372d131799f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b68eb3436824dce87c3e372d131799f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 44939 } }
I20260809 15:07:46.347348 1238 leader_election.cc:302] T 00000000000000000000000000000000 P 4b68eb3436824dce87c3e372d131799f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b68eb3436824dce87c3e372d131799f; no voters:
I20260809 15:07:46.347910 1238 leader_election.cc:288] T 00000000000000000000000000000000 P 4b68eb3436824dce87c3e372d131799f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:07:46.348017 1244 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b68eb3436824dce87c3e372d131799f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:07:46.348659 1244 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b68eb3436824dce87c3e372d131799f [term 1 LEADER]: Becoming Leader. State: Replica: 4b68eb3436824dce87c3e372d131799f, State: Running, Role: LEADER
I20260809 15:07:46.349145 1244 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b68eb3436824dce87c3e372d131799f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b68eb3436824dce87c3e372d131799f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 44939 } }
I20260809 15:07:46.349905 1238 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b68eb3436824dce87c3e372d131799f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:07:46.351732 1245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b68eb3436824dce87c3e372d131799f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b68eb3436824dce87c3e372d131799f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b68eb3436824dce87c3e372d131799f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 44939 } } }
I20260809 15:07:46.351706 1246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b68eb3436824dce87c3e372d131799f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b68eb3436824dce87c3e372d131799f. Latest consensus state: current_term: 1 leader_uuid: "4b68eb3436824dce87c3e372d131799f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b68eb3436824dce87c3e372d131799f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 44939 } } }
I20260809 15:07:46.352123 1245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b68eb3436824dce87c3e372d131799f [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:46.352198 1246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b68eb3436824dce87c3e372d131799f [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:46.355188 1255 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:07:46.360630 1255 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:07:46.368250 1255 catalog_manager.cc:1347] Generated new cluster ID: ba907c8438fc44658400048ea461c9d4
I20260809 15:07:46.368351 1255 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:07:46.389238 1255 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:07:46.390537 1255 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:07:46.398710 1255 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b68eb3436824dce87c3e372d131799f: Generated new TSK 0
I20260809 15:07:46.399483 1255 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:07:46.851300 1241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:46.425995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34554 (local address 127.24.183.254:44939)
0809 15:07:46.426193 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:46.426200 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:46.426558 (+ 358us) server_negotiation.cc:407] Connection header received
0809 15:07:46.426774 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:46.426781 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:46.426886 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:46.427039 (+ 153us) server_negotiation.cc:226] Negotiated authn=JWT
0809 15:07:46.427445 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:46.428606 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:46.430003 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:46.430691 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:46.430810 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:46.431230 (+ 420us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 15:07:46.850566 (+419336us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 15:07:46.851052 (+ 486us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34554: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":57}
W20260809 15:07:46.851416 1241 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34554: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 15:07:46.851543 1280 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:07:46.425874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:44939 (local address 127.0.0.1:34554)
0809 15:07:46.426470 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:46.426486 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:46.426600 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:46.427036 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:46.427039 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:46.427048 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0809 15:07:46.427284 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:46.427292 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:46.428731 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:46.428735 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:46.429871 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:46.429882 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:46.430843 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:46.430847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:46.430963 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:46.430995 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 15:07:46.850884 (+419889us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 15:07:46.851199 (+ 315us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.183.254:44939: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":314,"threads_started":1}
I20260809 15:07:46.852669 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 1133
I20260809 15:07:46.853807 1231 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:07:46.959420 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 1133
2026-08-09T15:07:46Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1134 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (417 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (415 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:07:47 dist-test-slave-nj21 krb5kdc[1302](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:07:47 dist-test-slave-nj21 krb5kdc[1302](info): set up 2 sockets
Aug 09 15:07:47 dist-test-slave-nj21 krb5kdc[1302](info): commencing operation
krb5kdc: starting...
W20260809 15:07:49.009286 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:07:49 dist-test-slave-nj21 krb5kdc[1302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:07:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:07:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
I20260809 15:07:49.233438 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:43331
--webserver_interface=127.24.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:43723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:43331
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:49.560611 1324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:49.560875 1324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:49.561009 1324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:49.567977 1324 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:07:49.568076 1324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:49.568154 1324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:49.568233 1324 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:07:49.568295 1324 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:07:49.577759 1324 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:43723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:43331
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:43331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:49.578997 1324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:49.580307 1324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:49.591888 1337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:49.591945 1335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:49.592068 1332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:49.592808 1324 server_base.cc:1034] running on GCE node
I20260809 15:07:49.593550 1324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:49.594836 1324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:49.596033 1324 hybrid_clock.cc:648] HybridClock initialized: now 1786288069595977 us; error 43 us; skew 500 ppm
I20260809 15:07:49.596354 1324 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:49 dist-test-slave-nj21 krb5kdc[1302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288069, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:49.604106 1324 init.cc:377] Logged in from keytab as kudu/127.24.183.254@KRBTEST.COM (short username kudu)
I20260809 15:07:49.605579 1324 webserver.cc:460] Webserver started at http://127.24.183.254:33621/ using document root <none> and password file <none>
I20260809 15:07:49.606269 1324 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:49.606388 1324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:49.606705 1324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:49.609732 1324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "880e52a638e0410997d7bc085da100bd"
format_stamp: "Formatted at 2026-08-09 15:07:49 on dist-test-slave-nj21"
server_key: "0244b60f461a969694c6cafe0c201ded"
server_key_iv: "37a5695d02993a20c0baa06214b044de"
server_key_version: "encryptionkey@0"
I20260809 15:07:49.610502 1324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "880e52a638e0410997d7bc085da100bd"
format_stamp: "Formatted at 2026-08-09 15:07:49 on dist-test-slave-nj21"
server_key: "0244b60f461a969694c6cafe0c201ded"
server_key_iv: "37a5695d02993a20c0baa06214b044de"
server_key_version: "encryptionkey@0"
I20260809 15:07:49.615628 1324 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 15:07:49.618976 1352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:49.620277 1324 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:07:49.620442 1324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "880e52a638e0410997d7bc085da100bd"
format_stamp: "Formatted at 2026-08-09 15:07:49 on dist-test-slave-nj21"
server_key: "0244b60f461a969694c6cafe0c201ded"
server_key_iv: "37a5695d02993a20c0baa06214b044de"
server_key_version: "encryptionkey@0"
I20260809 15:07:49.620618 1324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:49.634310 1324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:49.638417 1324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:49.638671 1324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:49.656296 1324 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:43331
I20260809 15:07:49.656339 1430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:43331 every 8 connection(s)
I20260809 15:07:49.657611 1324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:07:49.661226 1431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:49.663455 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 1324
I20260809 15:07:49.663690 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:07:49.664531 25311 external_mini_cluster.cc:1428] Setting key 286e9c256c30bcbcbeece0d4260a37c7
I20260809 15:07:49.673610 1431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 880e52a638e0410997d7bc085da100bd: Bootstrap starting.
I20260809 15:07:49.677541 1431 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 880e52a638e0410997d7bc085da100bd: Neither blocks nor log segments found. Creating new log.
Aug 09 15:07:49 dist-test-slave-nj21 krb5kdc[1302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:49.678828 1431 log.cc:824] T 00000000000000000000000000000000 P 880e52a638e0410997d7bc085da100bd: Log is configured to *not* fsync() on all Append() calls
I20260809 15:07:49.682876 1431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 880e52a638e0410997d7bc085da100bd: No bootstrap required, opened a new log
I20260809 15:07:49.688081 1438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:49.666534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38232 (local address 127.24.183.254:43331)
0809 15:07:49.667346 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:49.667356 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:49.667397 (+ 41us) server_negotiation.cc:407] Connection header received
0809 15:07:49.669115 (+ 1718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:49.669138 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:49.669503 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:49.669829 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:49.671565 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:49.672882 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:49.674592 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:49.675504 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:49.675711 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:49.680081 (+ 4370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:49.680089 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:49.680103 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:49.680143 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:49.683922 (+ 3779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:49.684766 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:49.684775 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:49.684783 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:49.684974 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:49.685351 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:49.685360 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:49.685364 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:49.685911 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:49.686208 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:49.686633 (+ 425us) server_negotiation.cc:299] Negotiation successful
0809 15:07:49.687024 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":376,"threads_started":1}
I20260809 15:07:49.691573 1431 raft_consensus.cc:348] T 00000000000000000000000000000000 P 880e52a638e0410997d7bc085da100bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "880e52a638e0410997d7bc085da100bd" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 43331 } }
I20260809 15:07:49.691963 1431 raft_consensus.cc:374] T 00000000000000000000000000000000 P 880e52a638e0410997d7bc085da100bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:49.692068 1431 raft_consensus.cc:729] T 00000000000000000000000000000000 P 880e52a638e0410997d7bc085da100bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 880e52a638e0410997d7bc085da100bd, State: Initialized, Role: FOLLOWER
I20260809 15:07:49.692672 1431 consensus_queue.cc:260] T 00000000000000000000000000000000 P 880e52a638e0410997d7bc085da100bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "880e52a638e0410997d7bc085da100bd" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 43331 } }
I20260809 15:07:49.692929 1431 raft_consensus.cc:388] T 00000000000000000000000000000000 P 880e52a638e0410997d7bc085da100bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:07:49.693068 1431 raft_consensus.cc:482] T 00000000000000000000000000000000 P 880e52a638e0410997d7bc085da100bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:07:49.693223 1431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 880e52a638e0410997d7bc085da100bd [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:49.695274 1431 raft_consensus.cc:504] T 00000000000000000000000000000000 P 880e52a638e0410997d7bc085da100bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "880e52a638e0410997d7bc085da100bd" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 43331 } }
I20260809 15:07:49.695779 1431 leader_election.cc:302] T 00000000000000000000000000000000 P 880e52a638e0410997d7bc085da100bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 880e52a638e0410997d7bc085da100bd; no voters:
I20260809 15:07:49.696313 1431 leader_election.cc:288] T 00000000000000000000000000000000 P 880e52a638e0410997d7bc085da100bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:07:49.696427 1440 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 880e52a638e0410997d7bc085da100bd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:07:49.697093 1440 raft_consensus.cc:686] T 00000000000000000000000000000000 P 880e52a638e0410997d7bc085da100bd [term 1 LEADER]: Becoming Leader. State: Replica: 880e52a638e0410997d7bc085da100bd, State: Running, Role: LEADER
I20260809 15:07:49.697623 1440 consensus_queue.cc:237] T 00000000000000000000000000000000 P 880e52a638e0410997d7bc085da100bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "880e52a638e0410997d7bc085da100bd" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 43331 } }
I20260809 15:07:49.698261 1431 sys_catalog.cc:564] T 00000000000000000000000000000000 P 880e52a638e0410997d7bc085da100bd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:07:49.700270 1442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 880e52a638e0410997d7bc085da100bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 880e52a638e0410997d7bc085da100bd. Latest consensus state: current_term: 1 leader_uuid: "880e52a638e0410997d7bc085da100bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "880e52a638e0410997d7bc085da100bd" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 43331 } } }
I20260809 15:07:49.700692 1442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 880e52a638e0410997d7bc085da100bd [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:49.701211 1441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 880e52a638e0410997d7bc085da100bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "880e52a638e0410997d7bc085da100bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "880e52a638e0410997d7bc085da100bd" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 43331 } } }
I20260809 15:07:49.701597 1441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 880e52a638e0410997d7bc085da100bd [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:49.701602 1448 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:07:49.707456 1448 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:07:49.715267 1448 catalog_manager.cc:1347] Generated new cluster ID: 7a1fe4a9bf5641818129389c70e3f324
I20260809 15:07:49.715381 1448 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:07:49.752730 1448 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:07:49.754025 1448 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:07:49.763733 1448 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 880e52a638e0410997d7bc085da100bd: Generated new TSK 0
I20260809 15:07:49.764506 1448 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab.
Entry for principal kudu/127.24.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab.
I20260809 15:07:49.824836 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.193:0
--local_ip_for_outbound_sockets=127.24.183.193
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:43331
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:43723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:50.150038 1467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:50.150309 1467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:50.150442 1467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:50.157637 1467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:50.157763 1467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:50.157909 1467 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.193
I20260809 15:07:50.167431 1467 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:43723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:43331
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.193
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:50.168735 1467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:50.170138 1467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:50.182901 1479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:50.182950 1482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:50.182976 1478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:50.183729 1467 server_base.cc:1034] running on GCE node
I20260809 15:07:50.184306 1467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:50.185186 1467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:50.186404 1467 hybrid_clock.cc:648] HybridClock initialized: now 1786288070186352 us; error 41 us; skew 500 ppm
I20260809 15:07:50.186719 1467 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:50 dist-test-slave-nj21 krb5kdc[1302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288070, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:50.189808 1467 init.cc:377] Logged in from keytab as kudu/127.24.183.193@KRBTEST.COM (short username kudu)
I20260809 15:07:50.191219 1467 webserver.cc:460] Webserver started at http://127.24.183.193:36645/ using document root <none> and password file <none>
I20260809 15:07:50.191912 1467 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:50.192018 1467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:50.192288 1467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:50.195130 1467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-0/data/instance:
uuid: "334da059e64a46098180c7959439eb39"
format_stamp: "Formatted at 2026-08-09 15:07:50 on dist-test-slave-nj21"
server_key: "c42c3c6eeac1a13c76fe3386d571598c"
server_key_iv: "ba6eab08b9ff75b48d03149643ff71a1"
server_key_version: "encryptionkey@0"
I20260809 15:07:50.195945 1467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance:
uuid: "334da059e64a46098180c7959439eb39"
format_stamp: "Formatted at 2026-08-09 15:07:50 on dist-test-slave-nj21"
server_key: "c42c3c6eeac1a13c76fe3386d571598c"
server_key_iv: "ba6eab08b9ff75b48d03149643ff71a1"
server_key_version: "encryptionkey@0"
I20260809 15:07:50.201040 1467 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260809 15:07:50.204610 1494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:50.205890 1467 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:07:50.206027 1467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-0/wal
uuid: "334da059e64a46098180c7959439eb39"
format_stamp: "Formatted at 2026-08-09 15:07:50 on dist-test-slave-nj21"
server_key: "c42c3c6eeac1a13c76fe3386d571598c"
server_key_iv: "ba6eab08b9ff75b48d03149643ff71a1"
server_key_version: "encryptionkey@0"
I20260809 15:07:50.206189 1467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:50.224133 1467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:50.228173 1467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:50.228410 1467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:50.229269 1467 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:07:50.230556 1467 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:07:50.230629 1467 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:50.230734 1467 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:07:50.230778 1467 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:50.261322 1467 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.193:46347
I20260809 15:07:50.261355 1680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.193:46347 every 8 connection(s)
I20260809 15:07:50.262552 1467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
I20260809 15:07:50.266048 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 1467
I20260809 15:07:50.266243 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance
I20260809 15:07:50.267045 25311 external_mini_cluster.cc:1428] Setting key ee061644c0eb8b165cd419acff5b73a6
Aug 09 15:07:50 dist-test-slave-nj21 krb5kdc[1302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288070, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:50.282640 1683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:50.265098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:46439 (local address 127.24.183.254:43331)
0809 15:07:50.265488 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:50.265493 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:50.266669 (+ 1176us) server_negotiation.cc:407] Connection header received
0809 15:07:50.268782 (+ 2113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:50.268789 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:50.268871 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:50.268972 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:50.270476 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:50.271501 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:50.273132 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:50.273665 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:50.273783 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:50.277009 (+ 3226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:50.277014 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:50.277019 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:50.277055 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:50.279660 (+ 2605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:50.280410 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:50.280415 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:50.280420 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:50.280570 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:50.280976 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:50.280980 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:50.280983 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:50.281319 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:50.281468 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:50.282086 (+ 618us) server_negotiation.cc:299] Negotiation successful
0809 15:07:50.282302 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":216,"threads_started":1}
I20260809 15:07:50.283486 1685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:50.265666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:43331 (local address 127.24.183.193:46439)
0809 15:07:50.266511 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:50.266559 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:50.268503 (+ 1944us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:50.269227 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:50.269235 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:50.269744 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:50.270299 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:50.270317 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:50.271663 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:50.271668 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:50.272985 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:50.272998 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:50.273849 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:50.273854 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:50.274047 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:50.274672 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:50.274699 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:50.276799 (+ 2100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:50.279824 (+ 3025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:50.279830 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:50.279842 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:50.280270 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:50.280678 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:50.280682 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:50.280684 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:50.280866 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:50.281432 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:50.281437 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:50.281733 (+ 296us) client_negotiation.cc:769] Sending connection context
0809 15:07:50.281947 (+ 214us) client_negotiation.cc:240] Negotiation successful
0809 15:07:50.282348 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":335,"threads_started":1}
I20260809 15:07:50.284986 1681 heartbeater.cc:347] Connected to a master server at 127.24.183.254:43331
I20260809 15:07:50.285302 1681 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:50.286175 1681 heartbeater.cc:511] Master 127.24.183.254:43331 requested a full tablet report, sending...
I20260809 15:07:50.288511 1376 ts_manager.cc:194] Registered new tserver with Master: 334da059e64a46098180c7959439eb39 (127.24.183.193:46347)
I20260809 15:07:50.289954 1376 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.193@KRBTEST.COM'} at 127.24.183.193:46439
WARNING: no policy specified for kudu/127.24.183.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab.
Entry for principal kudu/127.24.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab.
I20260809 15:07:50.320781 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.194:0
--local_ip_for_outbound_sockets=127.24.183.194
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:43331
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:43723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:50.647053 1688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:50.647317 1688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:50.647415 1688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:50.654557 1688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:50.654663 1688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:50.654785 1688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.194
I20260809 15:07:50.669203 1688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:43723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:43331
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.194
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:50.670750 1688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:50.672426 1688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:50.686196 1700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:50.686203 1695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:50.686203 1693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:50.687047 1688 server_base.cc:1034] running on GCE node
I20260809 15:07:50.687557 1688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:50.688467 1688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:50.689661 1688 hybrid_clock.cc:648] HybridClock initialized: now 1786288070689616 us; error 37 us; skew 500 ppm
I20260809 15:07:50.690114 1688 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:50 dist-test-slave-nj21 krb5kdc[1302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288070, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:50.693431 1688 init.cc:377] Logged in from keytab as kudu/127.24.183.194@KRBTEST.COM (short username kudu)
I20260809 15:07:50.694813 1688 webserver.cc:460] Webserver started at http://127.24.183.194:39411/ using document root <none> and password file <none>
I20260809 15:07:50.695462 1688 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:50.695580 1688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:50.695926 1688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:50.698882 1688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-1/data/instance:
uuid: "2e18ae349813494eba39a4b2d1857798"
format_stamp: "Formatted at 2026-08-09 15:07:50 on dist-test-slave-nj21"
server_key: "0b07fe6fae37aa4b3a40fa4c44d42e0b"
server_key_iv: "979d951a7110fcfcbef53b90c4a66eaf"
server_key_version: "encryptionkey@0"
I20260809 15:07:50.699707 1688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance:
uuid: "2e18ae349813494eba39a4b2d1857798"
format_stamp: "Formatted at 2026-08-09 15:07:50 on dist-test-slave-nj21"
server_key: "0b07fe6fae37aa4b3a40fa4c44d42e0b"
server_key_iv: "979d951a7110fcfcbef53b90c4a66eaf"
server_key_version: "encryptionkey@0"
I20260809 15:07:50.704715 1688 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 15:07:50.708022 1708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:50.709364 1688 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 15:07:50.709539 1688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-1/wal
uuid: "2e18ae349813494eba39a4b2d1857798"
format_stamp: "Formatted at 2026-08-09 15:07:50 on dist-test-slave-nj21"
server_key: "0b07fe6fae37aa4b3a40fa4c44d42e0b"
server_key_iv: "979d951a7110fcfcbef53b90c4a66eaf"
server_key_version: "encryptionkey@0"
I20260809 15:07:50.709729 1688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:50.737535 1688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:50.741870 1688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:50.742169 1688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:50.743294 1688 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:07:50.744661 1688 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:07:50.744740 1688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:50.744858 1688 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:07:50.744922 1688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:50.775060 1688 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.194:40705
I20260809 15:07:50.775107 1889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.194:40705 every 8 connection(s)
I20260809 15:07:50.776330 1688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
I20260809 15:07:50.782179 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 1688
I20260809 15:07:50.782334 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance
I20260809 15:07:50.782754 25311 external_mini_cluster.cc:1428] Setting key 212dd445841d8061106ad0666efe0421
Aug 09 15:07:50 dist-test-slave-nj21 krb5kdc[1302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288070, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:50.795842 1683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:50.778737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:58343 (local address 127.24.183.254:43331)
0809 15:07:50.778914 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:50.778919 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:50.780054 (+ 1135us) server_negotiation.cc:407] Connection header received
0809 15:07:50.781872 (+ 1818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:50.781879 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:50.781969 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:50.782058 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:50.783585 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:50.784513 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:50.786014 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:50.786432 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:50.786539 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:50.790015 (+ 3476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:50.790022 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:50.790026 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:50.790063 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:50.792963 (+ 2900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:50.793670 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:50.793674 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:50.793677 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:50.793807 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:50.794241 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:50.794244 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:50.794246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:50.794502 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:50.794610 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:50.795377 (+ 767us) server_negotiation.cc:299] Negotiation successful
0809 15:07:50.795556 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 15:07:50.797111 1894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:50.779153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:43331 (local address 127.24.183.194:58343)
0809 15:07:50.779890 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:50.779937 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:50.781603 (+ 1666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:50.782311 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:50.782322 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:50.782855 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:50.783420 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:50.783438 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:50.784662 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:50.784666 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:50.785877 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:50.785889 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:50.786647 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:50.786655 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:50.786868 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:50.787473 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:50.787495 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:50.789797 (+ 2302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:50.793086 (+ 3289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:50.793092 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:50.793103 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:50.793560 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:50.793902 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:50.793906 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:50.793920 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:50.794148 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:50.794617 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:50.794625 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:50.795036 (+ 411us) client_negotiation.cc:769] Sending connection context
0809 15:07:50.795302 (+ 266us) client_negotiation.cc:240] Negotiation successful
0809 15:07:50.795816 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":281,"threads_started":1}
I20260809 15:07:50.798508 1892 heartbeater.cc:347] Connected to a master server at 127.24.183.254:43331
I20260809 15:07:50.798796 1892 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:50.799554 1892 heartbeater.cc:511] Master 127.24.183.254:43331 requested a full tablet report, sending...
I20260809 15:07:50.800962 1376 ts_manager.cc:194] Registered new tserver with Master: 2e18ae349813494eba39a4b2d1857798 (127.24.183.194:40705)
I20260809 15:07:50.801767 1376 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.194@KRBTEST.COM'} at 127.24.183.194:58343
WARNING: no policy specified for kudu/127.24.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab.
Entry for principal kudu/127.24.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab.
I20260809 15:07:50.832624 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.195:0
--local_ip_for_outbound_sockets=127.24.183.195
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:43331
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:43723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:07:51.293329 1681 heartbeater.cc:503] Master 127.24.183.254:43331 was elected leader, sending a full tablet report...
W20260809 15:07:51.365340 1898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:51.365625 1898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:51.365777 1898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:51.372826 1898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:51.372936 1898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:51.373085 1898 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.195
I20260809 15:07:51.382580 1898 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:43723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:43331
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.195
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:51.383873 1898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:51.385216 1898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:51.398118 1905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:51.398195 1906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:51.398641 1908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:51.399147 1898 server_base.cc:1034] running on GCE node
I20260809 15:07:51.399796 1898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:51.400738 1898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:51.401927 1898 hybrid_clock.cc:648] HybridClock initialized: now 1786288071401882 us; error 39 us; skew 500 ppm
I20260809 15:07:51.402220 1898 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:51 dist-test-slave-nj21 krb5kdc[1302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288071, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:51.405450 1898 init.cc:377] Logged in from keytab as kudu/127.24.183.195@KRBTEST.COM (short username kudu)
I20260809 15:07:51.406769 1898 webserver.cc:460] Webserver started at http://127.24.183.195:43091/ using document root <none> and password file <none>
I20260809 15:07:51.407451 1898 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:51.407563 1898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:51.407920 1898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:51.410907 1898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-2/data/instance:
uuid: "9368a6c2bbe24a159b677f17bccd150a"
format_stamp: "Formatted at 2026-08-09 15:07:51 on dist-test-slave-nj21"
server_key: "c85d0e3f8daed59a1a95bc4541e929d5"
server_key_iv: "63927bb5e32a8b2483dea54e52d2886b"
server_key_version: "encryptionkey@0"
I20260809 15:07:51.411741 1898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance:
uuid: "9368a6c2bbe24a159b677f17bccd150a"
format_stamp: "Formatted at 2026-08-09 15:07:51 on dist-test-slave-nj21"
server_key: "c85d0e3f8daed59a1a95bc4541e929d5"
server_key_iv: "63927bb5e32a8b2483dea54e52d2886b"
server_key_version: "encryptionkey@0"
I20260809 15:07:51.416769 1898 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 15:07:51.420065 1916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:51.421377 1898 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:07:51.421537 1898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-2/wal
uuid: "9368a6c2bbe24a159b677f17bccd150a"
format_stamp: "Formatted at 2026-08-09 15:07:51 on dist-test-slave-nj21"
server_key: "c85d0e3f8daed59a1a95bc4541e929d5"
server_key_iv: "63927bb5e32a8b2483dea54e52d2886b"
server_key_version: "encryptionkey@0"
I20260809 15:07:51.421720 1898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:51.446993 1898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:51.450717 1898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:51.450991 1898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:51.451934 1898 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:07:51.453218 1898 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:07:51.453297 1898 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:51.453421 1898 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:07:51.453476 1898 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:51.483958 1898 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.195:46045
I20260809 15:07:51.484055 2103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.195:46045 every 8 connection(s)
I20260809 15:07:51.485188 1898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
I20260809 15:07:51.487165 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 1898
I20260809 15:07:51.487344 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance
I20260809 15:07:51.487879 25311 external_mini_cluster.cc:1428] Setting key e2772415a784ffb030bf966f6bc303ff
Aug 09 15:07:51 dist-test-slave-nj21 krb5kdc[1302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288071, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:07:51.504698 2111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:51.487722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:58389 (local address 127.24.183.254:43331)
0809 15:07:51.488301 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:51.488308 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:51.489178 (+ 870us) server_negotiation.cc:407] Connection header received
0809 15:07:51.491291 (+ 2113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:51.491296 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:51.491407 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:51.491506 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:51.493048 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:51.493816 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:51.495293 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:51.495747 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:51.495857 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:51.499132 (+ 3275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:51.499137 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:51.499140 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:51.499205 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:51.501662 (+ 2457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:51.502315 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:51.502321 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:51.502324 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:51.502442 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:51.502889 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:51.502895 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:51.502898 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:51.503223 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:51.503357 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:51.504093 (+ 736us) server_negotiation.cc:299] Negotiation successful
0809 15:07:51.504318 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":254,"threads_started":1}
I20260809 15:07:51.505590 2112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:51.488250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:43331 (local address 127.24.183.195:58389)
0809 15:07:51.489028 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:51.489072 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:51.490962 (+ 1890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:51.491782 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:51.491790 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:51.492294 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:51.492886 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:51.492905 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:51.493966 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:51.493970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:51.495153 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:51.495171 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:51.495958 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:51.495967 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:51.496196 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:51.496785 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:51.496807 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:51.498943 (+ 2136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:51.501781 (+ 2838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:51.501795 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:51.501809 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:51.502204 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:51.502549 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:51.502554 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:51.502558 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:51.502787 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:51.503341 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:51.503349 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:51.503659 (+ 310us) client_negotiation.cc:769] Sending connection context
0809 15:07:51.503970 (+ 311us) client_negotiation.cc:240] Negotiation successful
0809 15:07:51.504492 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":325,"threads_started":1}
I20260809 15:07:51.506814 2108 heartbeater.cc:347] Connected to a master server at 127.24.183.254:43331
I20260809 15:07:51.507109 2108 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:51.507838 2108 heartbeater.cc:511] Master 127.24.183.254:43331 requested a full tablet report, sending...
I20260809 15:07:51.509194 1376 ts_manager.cc:194] Registered new tserver with Master: 9368a6c2bbe24a159b677f17bccd150a (127.24.183.195:46045)
I20260809 15:07:51.510087 1376 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.195@KRBTEST.COM'} at 127.24.183.195:58389
I20260809 15:07:51.514927 25311 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/krb5kdc/test-admin.keytab.
Aug 09 15:07:51 dist-test-slave-nj21 krb5kdc[1302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:51.545112 25311 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 15:07:51.550032 2123 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:07:51.547666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44924 (local address 127.24.183.193:46347)
0809 15:07:51.548312 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:51.548319 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:51.548349 (+ 30us) server_negotiation.cc:407] Connection header received
0809 15:07:51.548418 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:51.548425 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:51.548632 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:51.548761 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:51.549626 (+ 865us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44924: BlockingRecv error: recv got EOF from 127.0.0.1:44924 (error 108)
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":283,"threads_started":1}
W20260809 15:07:51.550041 25311 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/krb5kdc/krb5cc' of type FILE
Aug 09 15:07:51 dist-test-slave-nj21 krb5kdc[1302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:51.551525 25311 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 15:07:51 dist-test-slave-nj21 krb5kdc[1302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:51.552827 25311 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 15:07:51 dist-test-slave-nj21 krb5kdc[1302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.193@KRBTEST.COM
I20260809 15:07:51.567557 2123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:51.554454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44932 (local address 127.24.183.193:46347)
0809 15:07:51.554626 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:51.554632 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:51.554736 (+ 104us) server_negotiation.cc:407] Connection header received
0809 15:07:51.554894 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:51.554898 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:51.554955 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:51.555035 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:51.556010 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:51.557002 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:51.558204 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:51.558768 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:51.558878 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:51.562227 (+ 3349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:51.562241 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:51.562257 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:51.562298 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:51.565257 (+ 2959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:51.565840 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:51.565846 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:51.565849 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:51.565985 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:51.566289 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:51.566292 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:51.566294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:51.566638 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:51.566856 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:51.567085 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 15:07:51.567274 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260809 15:07:51.572721 2123 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:07:51.571214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44938 (local address 127.24.183.193:46347)
0809 15:07:51.571399 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:51.571404 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:51.571566 (+ 162us) server_negotiation.cc:407] Connection header received
0809 15:07:51.571659 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:51.571662 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:51.571747 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:51.571814 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:51.572520 (+ 706us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44938: BlockingRecv error: recv got EOF from 127.0.0.1:44938 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260809 15:07:51.573107 25311 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/krb5kdc/krb5cc' of type FILE
Aug 09 15:07:51 dist-test-slave-nj21 krb5kdc[1302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:51.574386 25311 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 15:07:51 dist-test-slave-nj21 krb5kdc[1302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:51.575567 25311 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 15:07:51 dist-test-slave-nj21 krb5kdc[1302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.193@KRBTEST.COM
I20260809 15:07:51.588272 2123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:51.576728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44946 (local address 127.24.183.193:46347)
0809 15:07:51.576871 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:51.576876 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:51.577154 (+ 278us) server_negotiation.cc:407] Connection header received
0809 15:07:51.577285 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:51.577290 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:51.577340 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:51.577413 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:51.578339 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:51.579004 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:51.580249 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:51.580690 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:51.580794 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:51.584029 (+ 3235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:51.584033 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:51.584036 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:51.584066 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:51.586284 (+ 2218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:51.586828 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:51.586832 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:51.586834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:51.586958 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:51.587222 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:51.587225 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:51.587227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:51.587458 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:51.587654 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:51.587883 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 15:07:51.588045 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260809 15:07:51.592023 2123 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:07:51.590654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44962 (local address 127.24.183.193:46347)
0809 15:07:51.590824 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:51.590829 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:51.590925 (+ 96us) server_negotiation.cc:407] Connection header received
0809 15:07:51.591055 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:51.591058 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:51.591122 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:51.591189 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:51.591805 (+ 616us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44962: BlockingRecv error: recv got EOF from 127.0.0.1:44962 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260809 15:07:51.592207 25311 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288021313853-25311-0/krb5kdc/krb5cc' of type FILE
Aug 09 15:07:51 dist-test-slave-nj21 krb5kdc[1302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:51.593569 25311 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 15:07:51 dist-test-slave-nj21 krb5kdc[1302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:51.594762 25311 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 15:07:51 dist-test-slave-nj21 krb5kdc[1302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.193@KRBTEST.COM
I20260809 15:07:51.607841 2123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:51.595924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44968 (local address 127.24.183.193:46347)
0809 15:07:51.596092 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:51.596097 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:51.596339 (+ 242us) server_negotiation.cc:407] Connection header received
0809 15:07:51.596483 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:51.596487 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:51.596537 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:51.596610 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:51.597551 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:51.598232 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:51.599459 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:51.599950 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:51.600056 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:51.603157 (+ 3101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:51.603161 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:51.603164 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:51.603193 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:51.605400 (+ 2207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:51.606002 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:51.606007 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:51.606011 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:51.606154 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:51.606483 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:51.606486 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:51.606488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:51.606757 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:51.607065 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:51.607308 (+ 243us) server_negotiation.cc:299] Negotiation successful
0809 15:07:51.607517 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 15:07:51.609205 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 1467
I20260809 15:07:51.619590 2123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:51.609874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44970 (local address 127.24.183.193:46347)
0809 15:07:51.610012 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:51.610017 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:51.610329 (+ 312us) server_negotiation.cc:407] Connection header received
0809 15:07:51.610469 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:51.610472 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:51.610523 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:51.610596 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:51.611402 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:51.612116 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:51.613283 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:51.613699 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:51.613789 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:51.615590 (+ 1801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:51.615594 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:51.615597 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:51.615623 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:51.617706 (+ 2083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:51.618225 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:51.618228 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:51.618230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:51.618337 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:51.618583 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:51.618585 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:51.618587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:51.618809 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:51.619003 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:51.619200 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 15:07:51.619342 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 15:07:51.620159 1671 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:07:51.706820 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 1467
I20260809 15:07:51.727928 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 1688
Aug 09 15:07:51 dist-test-slave-nj21 krb5kdc[1302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.194@KRBTEST.COM
I20260809 15:07:51.743227 2144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:51.728791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53600 (local address 127.24.183.194:40705)
0809 15:07:51.729314 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:51.729321 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:51.729347 (+ 26us) server_negotiation.cc:407] Connection header received
0809 15:07:51.729408 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:51.729416 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:51.729642 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:51.729785 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:51.730818 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:51.731920 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:51.733158 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:51.733702 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:51.733819 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:51.737723 (+ 3904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:51.737735 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:51.737750 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:51.737783 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:51.740468 (+ 2685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:51.741128 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:51.741135 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:51.741138 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:51.741272 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:51.741641 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:51.741645 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:51.741647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:51.742002 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:51.742319 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:51.742650 (+ 331us) server_negotiation.cc:299] Negotiation successful
0809 15:07:51.742919 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":248,"threads_started":1}
I20260809 15:07:51.744582 1884 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:07:51.827255 1892 heartbeater.cc:503] Master 127.24.183.254:43331 was elected leader, sending a full tablet report...
I20260809 15:07:51.828409 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 1688
W20260809 15:07:51.851378 1353 connection.cc:535] server connection from 127.24.183.194:58343 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 15:07:51.851635 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 1898
Aug 09 15:07:51 dist-test-slave-nj21 krb5kdc[1302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.195@KRBTEST.COM
I20260809 15:07:51.867096 2150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:51.852746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57364 (local address 127.24.183.195:46045)
0809 15:07:51.853442 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:51.853449 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:51.853480 (+ 31us) server_negotiation.cc:407] Connection header received
0809 15:07:51.853541 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:51.853548 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:51.853759 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:51.853884 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:51.854964 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:51.856061 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:51.857343 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:51.857912 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:51.858025 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:51.861978 (+ 3953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:51.861990 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:51.862006 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:51.862039 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:51.864668 (+ 2629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:51.865269 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:51.865276 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:51.865278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:51.865394 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:51.865741 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:51.865745 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:51.865747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:51.866100 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:51.866349 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:51.866545 (+ 196us) server_negotiation.cc:299] Negotiation successful
0809 15:07:51.866784 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":364,"threads_started":1}
I20260809 15:07:51.868391 2098 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:07:51.951985 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 1898
I20260809 15:07:51.972405 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 1324
I20260809 15:07:51.973562 1420 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:07:52.081441 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 1324
2026-08-09T15:07:52Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4291 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:07:52 dist-test-slave-nj21 krb5kdc[2156](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:07:52 dist-test-slave-nj21 krb5kdc[2156](info): set up 2 sockets
Aug 09 15:07:52 dist-test-slave-nj21 krb5kdc[2156](info): commencing operation
krb5kdc: starting...
W20260809 15:07:53.301733 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.007s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:07:53 dist-test-slave-nj21 krb5kdc[2156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:07:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:07:53Z Disabled control of system clock
WARNING: no policy specified for oryx/127.24.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.254.keytab.
Entry for principal oryx/127.24.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.254.keytab.
I20260809 15:07:53.514874 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:45957
--webserver_interface=127.24.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.254.keytab
--principal=oryx/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:37641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:45957
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:53.842080 2175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:53.842374 2175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:53.842511 2175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:53.849637 2175 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:07:53.849753 2175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:53.849836 2175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:53.849923 2175 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:07:53.849982 2175 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:07:53.859620 2175 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:37641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:45957
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.254.keytab
--principal=oryx/127.24.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:45957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:53.860975 2175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:53.862278 2175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:53.873289 2186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:53.873323 2188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:53.874009 2190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:53.874655 2175 server_base.cc:1034] running on GCE node
I20260809 15:07:53.875370 2175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:53.876956 2175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:53.878154 2175 hybrid_clock.cc:648] HybridClock initialized: now 1786288073878107 us; error 34 us; skew 500 ppm
I20260809 15:07:53.878482 2175 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:53 dist-test-slave-nj21 krb5kdc[2156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288073, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:53.881433 2175 init.cc:377] Logged in from keytab as oryx/127.24.183.254@KRBTEST.COM (short username oryx)
I20260809 15:07:53.882834 2175 webserver.cc:460] Webserver started at http://127.24.183.254:40617/ using document root <none> and password file <none>
I20260809 15:07:53.883476 2175 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:53.883591 2175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:53.883909 2175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:53.886773 2175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "2be4818e2d7f4e51ba9ff391f3aecb81"
format_stamp: "Formatted at 2026-08-09 15:07:53 on dist-test-slave-nj21"
server_key: "3dc64116d0e5578e442c1c4a4ad618bc"
server_key_iv: "ee663cb84ccb44a987ca682c0bcf7de9"
server_key_version: "encryptionkey@0"
I20260809 15:07:53.887495 2175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "2be4818e2d7f4e51ba9ff391f3aecb81"
format_stamp: "Formatted at 2026-08-09 15:07:53 on dist-test-slave-nj21"
server_key: "3dc64116d0e5578e442c1c4a4ad618bc"
server_key_iv: "ee663cb84ccb44a987ca682c0bcf7de9"
server_key_version: "encryptionkey@0"
I20260809 15:07:53.892367 2175 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 15:07:53.895654 2201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:53.896976 2175 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:07:53.897132 2175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "2be4818e2d7f4e51ba9ff391f3aecb81"
format_stamp: "Formatted at 2026-08-09 15:07:53 on dist-test-slave-nj21"
server_key: "3dc64116d0e5578e442c1c4a4ad618bc"
server_key_iv: "ee663cb84ccb44a987ca682c0bcf7de9"
server_key_version: "encryptionkey@0"
I20260809 15:07:53.897296 2175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:53.918898 2175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:53.922595 2175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:53.922864 2175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:53.940303 2175 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:45957
I20260809 15:07:53.940340 2276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:45957 every 8 connection(s)
I20260809 15:07:53.941612 2175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:07:53.944025 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 2175
I20260809 15:07:53.944252 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:07:53.945101 25311 external_mini_cluster.cc:1428] Setting key 17ec6b3cfacf7da46e06366060fc3296
I20260809 15:07:53.945926 2279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 15:07:53 dist-test-slave-nj21 krb5kdc[2156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM
I20260809 15:07:53.958340 2279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81: Bootstrap starting.
I20260809 15:07:53.961503 2279 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:53.962522 2279 log.cc:824] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81: Log is configured to *not* fsync() on all Append() calls
I20260809 15:07:53.965286 2279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81: No bootstrap required, opened a new log
I20260809 15:07:53.966135 2283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:53.947194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50124 (local address 127.24.183.254:45957)
0809 15:07:53.947986 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:53.947997 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:53.948037 (+ 40us) server_negotiation.cc:407] Connection header received
0809 15:07:53.949560 (+ 1523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:53.949581 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:53.949949 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:53.950309 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:53.951806 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:53.953151 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:53.954358 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:53.955058 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:53.955246 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:53.959282 (+ 4036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:53.959290 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:53.959304 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:53.959344 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:53.962232 (+ 2888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:53.962899 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:53.962906 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:53.962912 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:53.963091 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:53.963452 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:53.963456 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:53.963458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:53.964008 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:53.964306 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:53.964749 (+ 443us) server_negotiation.cc:299] Negotiation successful
0809 15:07:53.965139 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":363,"threads_started":1}
I20260809 15:07:53.973605 2279 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } }
I20260809 15:07:53.973943 2279 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:53.974045 2279 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2be4818e2d7f4e51ba9ff391f3aecb81, State: Initialized, Role: FOLLOWER
I20260809 15:07:53.974606 2279 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } }
I20260809 15:07:53.974830 2279 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:07:53.974957 2279 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:07:53.975117 2279 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:53.977541 2279 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } }
I20260809 15:07:53.978027 2279 leader_election.cc:302] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2be4818e2d7f4e51ba9ff391f3aecb81; no voters:
I20260809 15:07:53.978557 2279 leader_election.cc:288] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:07:53.978683 2285 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:07:53.979300 2285 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [term 1 LEADER]: Becoming Leader. State: Replica: 2be4818e2d7f4e51ba9ff391f3aecb81, State: Running, Role: LEADER
I20260809 15:07:53.979925 2285 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } }
I20260809 15:07:53.980522 2279 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:07:53.982528 2286 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } }
I20260809 15:07:53.982923 2286 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:53.982841 2287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2be4818e2d7f4e51ba9ff391f3aecb81. Latest consensus state: current_term: 1 leader_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } }
I20260809 15:07:53.983251 2287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:53.984982 2295 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:07:53.990453 2295 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:07:53.998775 2295 catalog_manager.cc:1347] Generated new cluster ID: 8879a6c828534eb6bb5186bd3f176cb4
I20260809 15:07:53.998945 2295 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:07:54.049916 2295 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:07:54.051296 2295 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:07:54.058768 2295 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81: Generated new TSK 0
I20260809 15:07:54.059639 2295 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.24.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.193.keytab.
Entry for principal oryx/127.24.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.193.keytab.
I20260809 15:07:54.134452 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.193:0
--local_ip_for_outbound_sockets=127.24.183.193
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:45957
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.193.keytab
--principal=oryx/127.24.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:37641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:54.479286 2314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:54.479565 2314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:54.479728 2314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:54.486994 2314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:54.487102 2314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:54.487246 2314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.193
I20260809 15:07:54.496711 2314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:37641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.193.keytab
--principal=oryx/127.24.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:45957
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.193
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:54.497925 2314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:54.499184 2314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:54.511583 2320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:54.511693 2321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:54.511653 2326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:54.512414 2314 server_base.cc:1034] running on GCE node
I20260809 15:07:54.512960 2314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:54.513803 2314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:54.514989 2314 hybrid_clock.cc:648] HybridClock initialized: now 1786288074514934 us; error 48 us; skew 500 ppm
I20260809 15:07:54.515267 2314 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:54 dist-test-slave-nj21 krb5kdc[2156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288074, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:54.518266 2314 init.cc:377] Logged in from keytab as oryx/127.24.183.193@KRBTEST.COM (short username oryx)
I20260809 15:07:54.519618 2314 webserver.cc:460] Webserver started at http://127.24.183.193:38651/ using document root <none> and password file <none>
I20260809 15:07:54.520251 2314 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:54.520385 2314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:54.520663 2314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:54.523386 2314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-0/data/instance:
uuid: "cfc40b3492174e7cb2dfd14d21e72582"
format_stamp: "Formatted at 2026-08-09 15:07:54 on dist-test-slave-nj21"
server_key: "62283c587478ca97c94d378438535804"
server_key_iv: "04e5e1f2323291845b82c36c2c03437e"
server_key_version: "encryptionkey@0"
I20260809 15:07:54.524223 2314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance:
uuid: "cfc40b3492174e7cb2dfd14d21e72582"
format_stamp: "Formatted at 2026-08-09 15:07:54 on dist-test-slave-nj21"
server_key: "62283c587478ca97c94d378438535804"
server_key_iv: "04e5e1f2323291845b82c36c2c03437e"
server_key_version: "encryptionkey@0"
I20260809 15:07:54.528997 2314 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260809 15:07:54.532302 2341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:54.533468 2314 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 15:07:54.533618 2314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-0/wal
uuid: "cfc40b3492174e7cb2dfd14d21e72582"
format_stamp: "Formatted at 2026-08-09 15:07:54 on dist-test-slave-nj21"
server_key: "62283c587478ca97c94d378438535804"
server_key_iv: "04e5e1f2323291845b82c36c2c03437e"
server_key_version: "encryptionkey@0"
I20260809 15:07:54.533794 2314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:54.561273 2314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:54.565393 2314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:54.565665 2314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:54.566603 2314 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:07:54.567935 2314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:07:54.568022 2314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:54.568142 2314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:07:54.568205 2314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:54.598629 2314 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.193:44531
I20260809 15:07:54.598673 2528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.193:44531 every 8 connection(s)
I20260809 15:07:54.599864 2314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
I20260809 15:07:54.606676 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 2314
I20260809 15:07:54.606895 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance
I20260809 15:07:54.607743 25311 external_mini_cluster.cc:1428] Setting key 480216725e52e0bde3671dae1279722e
Aug 09 15:07:54 dist-test-slave-nj21 krb5kdc[2156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288074, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM
I20260809 15:07:54.626458 2532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:54.602314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:34117 (local address 127.24.183.254:45957)
0809 15:07:54.602758 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:54.602763 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:54.603664 (+ 901us) server_negotiation.cc:407] Connection header received
0809 15:07:54.605473 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:54.605478 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:54.605562 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:54.605660 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:54.607489 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:54.608344 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:54.610224 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:54.610724 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:54.610833 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:54.620714 (+ 9881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:54.620719 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:54.620722 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:54.620759 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:54.623270 (+ 2511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:54.624186 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:54.624190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:54.624192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:54.624319 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:54.624805 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:54.624808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:54.624810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:54.625067 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:54.625174 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:54.626010 (+ 836us) server_negotiation.cc:299] Negotiation successful
0809 15:07:54.626185 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":232,"threads_started":1}
I20260809 15:07:54.627938 2533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:54.602832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.24.183.193:34117)
0809 15:07:54.603510 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:54.603551 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:54.605211 (+ 1660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:54.605896 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:54.605905 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:54.606509 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:54.607307 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:54.607332 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:54.608504 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:54.608509 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:54.610039 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:54.610055 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:54.610967 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:54.610975 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:54.611246 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:54.617634 (+ 6388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:54.617667 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:54.620431 (+ 2764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:54.623439 (+ 3008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:54.623449 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:54.623466 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:54.624074 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:54.624432 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:54.624436 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:54.624439 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:54.624693 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:54.625192 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:54.625200 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:54.625617 (+ 417us) client_negotiation.cc:769] Sending connection context
0809 15:07:54.625901 (+ 284us) client_negotiation.cc:240] Negotiation successful
0809 15:07:54.626491 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":256,"threads_started":1}
I20260809 15:07:54.629520 2530 heartbeater.cc:347] Connected to a master server at 127.24.183.254:45957
I20260809 15:07:54.629877 2530 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:54.630820 2530 heartbeater.cc:511] Master 127.24.183.254:45957 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.24.183.194@KRBTEST.COM; defaulting to no policy
I20260809 15:07:54.633078 2223 ts_manager.cc:194] Registered new tserver with Master: cfc40b3492174e7cb2dfd14d21e72582 (127.24.183.193:44531)
I20260809 15:07:54.634588 2223 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.183.193@KRBTEST.COM'} at 127.24.183.193:34117
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.183.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.194.keytab.
Entry for principal oryx/127.24.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.194.keytab.
I20260809 15:07:54.660028 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.194:0
--local_ip_for_outbound_sockets=127.24.183.194
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:45957
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.194.keytab
--principal=oryx/127.24.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:37641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:54.984076 2538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:54.984385 2538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:54.984526 2538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:54.991699 2538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:54.991823 2538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:54.991983 2538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.194
I20260809 15:07:55.001474 2538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:37641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.194.keytab
--principal=oryx/127.24.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:45957
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.194
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:55.002722 2538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:55.004014 2538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:55.016569 2546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:55.016752 2545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:55.017060 2548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:55.017354 2538 server_base.cc:1034] running on GCE node
I20260809 15:07:55.017906 2538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:55.018761 2538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:55.019950 2538 hybrid_clock.cc:648] HybridClock initialized: now 1786288075019902 us; error 38 us; skew 500 ppm
I20260809 15:07:55.020232 2538 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:55 dist-test-slave-nj21 krb5kdc[2156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288075, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:55.023265 2538 init.cc:377] Logged in from keytab as oryx/127.24.183.194@KRBTEST.COM (short username oryx)
I20260809 15:07:55.024611 2538 webserver.cc:460] Webserver started at http://127.24.183.194:43313/ using document root <none> and password file <none>
I20260809 15:07:55.025206 2538 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:55.025315 2538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:55.025570 2538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:55.028483 2538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-1/data/instance:
uuid: "10025c6198174f61b5495c94ea3a510e"
format_stamp: "Formatted at 2026-08-09 15:07:55 on dist-test-slave-nj21"
server_key: "f6862c29cd973827d277b573997f04e7"
server_key_iv: "8ce231dc9f80243f757bbd8858e5dd23"
server_key_version: "encryptionkey@0"
I20260809 15:07:55.029196 2538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance:
uuid: "10025c6198174f61b5495c94ea3a510e"
format_stamp: "Formatted at 2026-08-09 15:07:55 on dist-test-slave-nj21"
server_key: "f6862c29cd973827d277b573997f04e7"
server_key_iv: "8ce231dc9f80243f757bbd8858e5dd23"
server_key_version: "encryptionkey@0"
I20260809 15:07:55.034027 2538 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 15:07:55.037286 2558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:55.038573 2538 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 15:07:55.038723 2538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-1/wal
uuid: "10025c6198174f61b5495c94ea3a510e"
format_stamp: "Formatted at 2026-08-09 15:07:55 on dist-test-slave-nj21"
server_key: "f6862c29cd973827d277b573997f04e7"
server_key_iv: "8ce231dc9f80243f757bbd8858e5dd23"
server_key_version: "encryptionkey@0"
I20260809 15:07:55.038903 2538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:55.058229 2538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:55.062340 2538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:55.062611 2538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:55.063531 2538 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:07:55.064864 2538 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:07:55.064950 2538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:55.065074 2538 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:07:55.065126 2538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:55.095386 2538 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.194:46409
I20260809 15:07:55.095439 2750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.194:46409 every 8 connection(s)
I20260809 15:07:55.096678 2538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
I20260809 15:07:55.098811 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 2538
I20260809 15:07:55.098982 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance
I20260809 15:07:55.099521 25311 external_mini_cluster.cc:1428] Setting key dcac0603e7bd120df85d9f59b3552ecd
Aug 09 15:07:55 dist-test-slave-nj21 krb5kdc[2156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288075, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM
I20260809 15:07:55.116403 2532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:55.099125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:45479 (local address 127.24.183.254:45957)
0809 15:07:55.099353 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:55.099360 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:55.100648 (+ 1288us) server_negotiation.cc:407] Connection header received
0809 15:07:55.102646 (+ 1998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:55.102651 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:55.102735 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:55.102842 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:55.104297 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:55.105214 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:55.106675 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:55.107248 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:55.107368 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:55.110813 (+ 3445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:55.110820 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:55.110824 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:55.110864 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:55.113449 (+ 2585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:55.114245 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:55.114248 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:55.114251 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:55.114371 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:55.114798 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:55.114800 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:55.114802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:55.115061 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:55.115168 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:55.115970 (+ 802us) server_negotiation.cc:299] Negotiation successful
0809 15:07:55.116146 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 15:07:55.117738 2755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:55.099629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.24.183.194:45479)
0809 15:07:55.100496 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:55.100542 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:55.102375 (+ 1833us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:55.103030 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:55.103038 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:55.103541 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:55.104138 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:55.104155 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:55.105331 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:55.105334 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:55.106538 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:55.106550 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:55.107461 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:55.107467 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:55.107665 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:55.108321 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:55.108347 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:55.110575 (+ 2228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:55.113586 (+ 3011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:55.113594 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:55.113609 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:55.114146 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:55.114470 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:55.114474 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:55.114477 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:55.114708 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:55.115191 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:55.115199 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:55.115578 (+ 379us) client_negotiation.cc:769] Sending connection context
0809 15:07:55.115888 (+ 310us) client_negotiation.cc:240] Negotiation successful
0809 15:07:55.116378 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":327,"threads_started":1}
I20260809 15:07:55.119168 2752 heartbeater.cc:347] Connected to a master server at 127.24.183.254:45957
I20260809 15:07:55.119480 2752 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:55.120419 2752 heartbeater.cc:511] Master 127.24.183.254:45957 requested a full tablet report, sending...
I20260809 15:07:55.122018 2223 ts_manager.cc:194] Registered new tserver with Master: 10025c6198174f61b5495c94ea3a510e (127.24.183.194:46409)
I20260809 15:07:55.122818 2223 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.183.194@KRBTEST.COM'} at 127.24.183.194:45479
WARNING: no policy specified for oryx/127.24.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.195.keytab.
Entry for principal oryx/127.24.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.195.keytab.
I20260809 15:07:55.153062 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.195:0
--local_ip_for_outbound_sockets=127.24.183.195
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:45957
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.195.keytab
--principal=oryx/127.24.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:37641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:55.482995 2761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:55.483248 2761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:55.483364 2761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:55.490571 2761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:55.490669 2761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:55.490785 2761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.195
I20260809 15:07:55.500142 2761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:37641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.195.keytab
--principal=oryx/127.24.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:45957
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.195
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:55.501394 2761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:55.502696 2761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:55.515332 2772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:55.515391 2768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:55.515372 2769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:55.516203 2761 server_base.cc:1034] running on GCE node
I20260809 15:07:55.516779 2761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:55.517639 2761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:55.518831 2761 hybrid_clock.cc:648] HybridClock initialized: now 1786288075518777 us; error 47 us; skew 500 ppm
I20260809 15:07:55.519112 2761 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:55 dist-test-slave-nj21 krb5kdc[2156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288075, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:55.522061 2761 init.cc:377] Logged in from keytab as oryx/127.24.183.195@KRBTEST.COM (short username oryx)
I20260809 15:07:55.523352 2761 webserver.cc:460] Webserver started at http://127.24.183.195:36527/ using document root <none> and password file <none>
I20260809 15:07:55.524067 2761 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:55.524187 2761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:55.524477 2761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:55.527310 2761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-2/data/instance:
uuid: "2ed67a7ab72e4760917f53986f1a774a"
format_stamp: "Formatted at 2026-08-09 15:07:55 on dist-test-slave-nj21"
server_key: "f313318271435ef0f83e6396b1356c66"
server_key_iv: "005b5473dc8db234a59e8e4893770ef1"
server_key_version: "encryptionkey@0"
I20260809 15:07:55.528096 2761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance:
uuid: "2ed67a7ab72e4760917f53986f1a774a"
format_stamp: "Formatted at 2026-08-09 15:07:55 on dist-test-slave-nj21"
server_key: "f313318271435ef0f83e6396b1356c66"
server_key_iv: "005b5473dc8db234a59e8e4893770ef1"
server_key_version: "encryptionkey@0"
I20260809 15:07:55.533109 2761 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 15:07:55.536453 2781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:55.537873 2761 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 15:07:55.538031 2761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-2/wal
uuid: "2ed67a7ab72e4760917f53986f1a774a"
format_stamp: "Formatted at 2026-08-09 15:07:55 on dist-test-slave-nj21"
server_key: "f313318271435ef0f83e6396b1356c66"
server_key_iv: "005b5473dc8db234a59e8e4893770ef1"
server_key_version: "encryptionkey@0"
I20260809 15:07:55.538224 2761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:55.559471 2761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:55.563366 2761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:55.563621 2761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:55.564637 2761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:07:55.565949 2761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:07:55.566027 2761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:55.566145 2761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:07:55.566197 2761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:55.597088 2761 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.195:40011
I20260809 15:07:55.597129 2961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.195:40011 every 8 connection(s)
I20260809 15:07:55.598424 2761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
I20260809 15:07:55.605437 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 2761
I20260809 15:07:55.605597 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance
I20260809 15:07:55.606051 25311 external_mini_cluster.cc:1428] Setting key d9391ba85b6974dad21449bc9b1f464c
Aug 09 15:07:55 dist-test-slave-nj21 krb5kdc[2156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288075, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM
I20260809 15:07:55.617903 2532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:55.601067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:56217 (local address 127.24.183.254:45957)
0809 15:07:55.601237 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:55.601242 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:55.602730 (+ 1488us) server_negotiation.cc:407] Connection header received
0809 15:07:55.604616 (+ 1886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:55.604622 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:55.604698 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:55.604789 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:55.606303 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:55.607209 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:55.608878 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:55.609314 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:55.609450 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:55.612808 (+ 3358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:55.612813 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:55.612816 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:55.612851 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:55.615314 (+ 2463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:55.616048 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:55.616051 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:55.616054 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:55.616185 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:55.616524 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:55.616527 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:55.616529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:55.616784 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:55.616891 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:55.617468 (+ 577us) server_negotiation.cc:299] Negotiation successful
0809 15:07:55.617633 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 15:07:55.618848 2964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:55.601769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.24.183.195:56217)
0809 15:07:55.602575 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:55.602615 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:55.604324 (+ 1709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:55.605026 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:55.605034 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:55.605513 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:55.606097 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:55.606119 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:55.607359 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:55.607364 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:55.608735 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:55.608748 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:55.609554 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:55.609561 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:55.609779 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:55.610382 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:55.610404 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:55.612624 (+ 2220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:55.615445 (+ 2821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:55.615451 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:55.615462 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:55.615952 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:55.616263 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:55.616266 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:55.616268 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:55.616443 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:55.616880 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:55.616886 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:55.617170 (+ 284us) client_negotiation.cc:769] Sending connection context
0809 15:07:55.617386 (+ 216us) client_negotiation.cc:240] Negotiation successful
0809 15:07:55.617738 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":302,"threads_started":1}
I20260809 15:07:55.620175 2962 heartbeater.cc:347] Connected to a master server at 127.24.183.254:45957
I20260809 15:07:55.620496 2962 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:55.621249 2962 heartbeater.cc:511] Master 127.24.183.254:45957 requested a full tablet report, sending...
I20260809 15:07:55.622570 2223 ts_manager.cc:194] Registered new tserver with Master: 2ed67a7ab72e4760917f53986f1a774a (127.24.183.195:40011)
I20260809 15:07:55.623354 2223 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.183.195@KRBTEST.COM'} at 127.24.183.195:56217
I20260809 15:07:55.632929 25311 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:07:55.637593 2530 heartbeater.cc:503] Master 127.24.183.254:45957 was elected leader, sending a full tablet report...
WARNING: no policy specified for oryx/127.24.183.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.183.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.183.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.253.keytab.
Entry for principal oryx/127.24.183.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.253.keytab.
I20260809 15:07:55.684787 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.253:39987
--webserver_interface=127.24.183.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.253.keytab
--principal=oryx/127.24.183.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:37641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:45957,127.24.183.253:39987
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:56.018230 2969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:56.018496 2969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:56.018625 2969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:56.025738 2969 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:07:56.025843 2969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:56.025924 2969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:56.025997 2969 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:07:56.026059 2969 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:07:56.035897 2969 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:37641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:45957,127.24.183.253:39987
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.253.keytab
--principal=oryx/127.24.183.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.253:39987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.183.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
W20260809 15:07:56.036233 2969 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.24.183.254:45957,127.24.183.253:39987'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 15:07:56.037328 2969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:56.038668 2969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:56.050210 2977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:56.050297 2975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:56.050295 2974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:56.051215 2969 server_base.cc:1034] running on GCE node
I20260809 15:07:56.052004 2969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:56.053287 2969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:56.054498 2969 hybrid_clock.cc:648] HybridClock initialized: now 1786288076054436 us; error 38 us; skew 500 ppm
I20260809 15:07:56.054795 2969 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:56 dist-test-slave-nj21 krb5kdc[2156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288076, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:56.057796 2969 init.cc:377] Logged in from keytab as oryx/127.24.183.253@KRBTEST.COM (short username oryx)
I20260809 15:07:56.059201 2969 webserver.cc:460] Webserver started at http://127.24.183.253:43255/ using document root <none> and password file <none>
I20260809 15:07:56.059971 2969 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:56.060092 2969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:56.060375 2969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:56.063218 2969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-1/data/instance:
uuid: "d835c269415b4f52b71dcbac3af1a081"
format_stamp: "Formatted at 2026-08-09 15:07:56 on dist-test-slave-nj21"
server_key: "ceff5dfa0dee62810316b9ca7f45332e"
server_key_iv: "2959be9e98d6da57f220cd5c31cc8d6e"
server_key_version: "encryptionkey@0"
I20260809 15:07:56.063973 2969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-1/wal/instance:
uuid: "d835c269415b4f52b71dcbac3af1a081"
format_stamp: "Formatted at 2026-08-09 15:07:56 on dist-test-slave-nj21"
server_key: "ceff5dfa0dee62810316b9ca7f45332e"
server_key_iv: "2959be9e98d6da57f220cd5c31cc8d6e"
server_key_version: "encryptionkey@0"
I20260809 15:07:56.068933 2969 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 15:07:56.072139 2989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:56.073374 2969 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 15:07:56.073520 2969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-1/wal
uuid: "d835c269415b4f52b71dcbac3af1a081"
format_stamp: "Formatted at 2026-08-09 15:07:56 on dist-test-slave-nj21"
server_key: "ceff5dfa0dee62810316b9ca7f45332e"
server_key_iv: "2959be9e98d6da57f220cd5c31cc8d6e"
server_key_version: "encryptionkey@0"
I20260809 15:07:56.073706 2969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:56.087714 2969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:56.091190 2969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:56.091421 2969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:56.109393 2969 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.253:39987
I20260809 15:07:56.109400 3076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.253:39987 every 8 connection(s)
I20260809 15:07:56.110670 2969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-1/data/info.pb
I20260809 15:07:56.114091 3077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:56.115152 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 2969
I20260809 15:07:56.123205 3077 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:07:56.125763 2752 heartbeater.cc:503] Master 127.24.183.254:45957 was elected leader, sending a full tablet report...
Aug 09 15:07:56 dist-test-slave-nj21 krb5kdc[2156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.183.253@KRBTEST.COM
Aug 09 15:07:56 dist-test-slave-nj21 krb5kdc[2156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288076, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.253@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM
I20260809 15:07:56.139456 3081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:56.117439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46798 (local address 127.24.183.253:39987)
0809 15:07:56.118192 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:56.118207 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:56.118255 (+ 48us) server_negotiation.cc:407] Connection header received
0809 15:07:56.119872 (+ 1617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:56.119898 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:56.120266 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:56.120616 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:56.122159 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.123378 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.124644 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.125504 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.125684 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:56.130845 (+ 5161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:56.130856 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:56.130872 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:56.130920 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:56.134802 (+ 3882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:56.135460 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:56.135473 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:56.135482 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:56.135703 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:56.136112 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:56.136115 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:56.136117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:56.136689 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:56.137011 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:56.137501 (+ 490us) server_negotiation.cc:299] Negotiation successful
0809 15:07:56.137937 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":340,"threads_started":1}
I20260809 15:07:56.142989 3083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:56.125571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.0.0.1:37250)
0809 15:07:56.126895 (+ 1324us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:56.126957 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:56.127095 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:56.128246 (+ 1151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:56.128254 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:56.129028 (+ 774us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:56.129302 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:56.129311 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.130689 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.130695 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:56.132511 (+ 1816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:56.132526 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.133395 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.133402 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:56.133593 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:56.134140 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:56.134168 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:56.136814 (+ 2646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:56.140325 (+ 3511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:56.140336 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:56.140343 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:56.140836 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:56.141223 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:56.141229 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:56.141232 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:56.141443 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:56.141974 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:56.141981 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:56.142234 (+ 253us) client_negotiation.cc:769] Sending connection context
0809 15:07:56.142450 (+ 216us) client_negotiation.cc:240] Negotiation successful
0809 15:07:56.142720 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1072,"thread_start_us":939,"threads_started":1}
I20260809 15:07:56.143261 3082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:56.125460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37250 (local address 127.24.183.254:45957)
0809 15:07:56.127875 (+ 2415us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:56.127879 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:56.127896 (+ 17us) server_negotiation.cc:407] Connection header received
0809 15:07:56.127962 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:56.127967 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:56.128042 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:56.128147 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:56.129457 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.130512 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.132668 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.133196 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.133298 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:56.137002 (+ 3704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:56.137008 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:56.137011 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:56.137044 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:56.140201 (+ 3157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:56.140971 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:56.140976 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:56.140978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:56.141110 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:56.141544 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:56.141548 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:56.141550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:56.141854 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:56.141971 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:56.142730 (+ 759us) server_negotiation.cc:299] Negotiation successful
0809 15:07:56.142911 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1273,"thread_start_us":1296,"threads_started":1}
I20260809 15:07:56.144804 3077 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:07:56 dist-test-slave-nj21 krb5kdc[2156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288076, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.253@KRBTEST.COM for oryx/127.24.183.253@KRBTEST.COM
I20260809 15:07:56.156157 3083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:56.145562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:39987 (local address 127.0.0.1:46806)
0809 15:07:56.145742 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:56.145760 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:56.145859 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:56.146104 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:56.146108 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:56.146448 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:56.146574 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:56.146581 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.147493 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.147497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:56.148411 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:56.148423 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.149076 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.149081 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:56.149231 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:56.149684 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:56.149700 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:56.151579 (+ 1879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:56.153993 (+ 2414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:56.153998 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:56.154002 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:56.154367 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:56.154669 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:56.154674 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:56.154677 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:56.154803 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:56.155266 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:56.155270 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:56.155420 (+ 150us) client_negotiation.cc:769] Sending connection context
0809 15:07:56.155531 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 15:07:56.155807 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 15:07:56.156307 3081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:56.145581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46806 (local address 127.24.183.253:39987)
0809 15:07:56.145734 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:56.145739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:56.145813 (+ 74us) server_negotiation.cc:407] Connection header received
0809 15:07:56.145951 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:56.145954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:56.146006 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:56.146076 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:56.146694 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.147362 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.148538 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.148941 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.149047 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:56.151756 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:56.151760 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:56.151764 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:56.151791 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:56.153879 (+ 2088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:56.154468 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:56.154472 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:56.154474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:56.154577 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:56.154902 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:56.154907 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:56.154910 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:56.155162 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:56.155286 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:56.155628 (+ 342us) server_negotiation.cc:299] Negotiation successful
0809 15:07:56.155973 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":99,"server-negotiator.queue_time_us":39}
I20260809 15:07:56.160882 3077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081: Bootstrap starting.
I20260809 15:07:56.164098 3077 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:56.165127 3077 log.cc:824] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081: Log is configured to *not* fsync() on all Append() calls
I20260809 15:07:56.167654 3077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081: No bootstrap required, opened a new log
I20260809 15:07:56.174607 3077 raft_consensus.cc:348] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } }
I20260809 15:07:56.174922 3077 raft_consensus.cc:374] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:56.175010 3077 raft_consensus.cc:729] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d835c269415b4f52b71dcbac3af1a081, State: Initialized, Role: FOLLOWER
I20260809 15:07:56.175527 3077 consensus_queue.cc:260] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } }
I20260809 15:07:56.176329 3085 sys_catalog.cc:455] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } } }
I20260809 15:07:56.176666 3085 sys_catalog.cc:458] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:07:56.177273 3077 sys_catalog.cc:564] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 15:07:56.185861 3105 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:07:56.186043 3105 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:07:56.189122 2969 master_runner.cc:428] Detected that this master d835c269415b4f52b71dcbac3af1a081 is joining an existing cluster
I20260809 15:07:56.189205 2969 master_runner.cc:432] Initiating AddMaster RPC to add 127.24.183.253:39987
I20260809 15:07:56.210456 3082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:56.192881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37254 (local address 127.24.183.254:45957)
0809 15:07:56.193071 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:56.193078 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:56.193843 (+ 765us) server_negotiation.cc:407] Connection header received
0809 15:07:56.194136 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:56.194142 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:56.194240 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:56.194320 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:56.196007 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.197230 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.199578 (+ 2348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.200303 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.200404 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:56.203549 (+ 3145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:56.203557 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:56.203562 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:56.203608 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:56.207170 (+ 3562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:56.207976 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:56.207982 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:56.207986 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:56.208142 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:56.208650 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:56.208656 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:56.208660 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:56.209061 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:56.209184 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:56.209850 (+ 666us) server_negotiation.cc:299] Negotiation successful
0809 15:07:56.210095 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 15:07:56.211530 3115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:56.192984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.0.0.1:37254)
0809 15:07:56.193727 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:56.193750 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:56.193966 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:56.194562 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:56.194567 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:56.195331 (+ 764us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:56.195814 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:56.195827 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.197421 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.197427 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:56.199401 (+ 1974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:56.199417 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.200845 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.200861 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:56.201115 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:56.201990 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:56.202020 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:56.203310 (+ 1290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:56.207327 (+ 4017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:56.207332 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:56.207336 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:56.207827 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:56.208298 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:56.208306 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:56.208310 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:56.208495 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:56.209289 (+ 794us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:56.209293 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:56.209526 (+ 233us) client_negotiation.cc:769] Sending connection context
0809 15:07:56.210713 (+ 1187us) client_negotiation.cc:240] Negotiation successful
0809 15:07:56.211048 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":450,"threads_started":1}
WARNING: no policy specified for oryx/127.24.183.252@KRBTEST.COM; defaulting to no policy
I20260809 15:07:56.225883 3116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:56.193963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:39987 (local address 127.0.0.1:46818)
0809 15:07:56.204953 (+ 10990us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:56.204972 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:56.205135 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:56.205892 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:56.205896 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:56.206570 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:56.206900 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:56.206912 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.208264 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.208269 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:56.209715 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:56.209729 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.211147 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.211154 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:56.211362 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:56.212201 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:56.212221 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:56.213159 (+ 938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:56.216947 (+ 3788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:56.216951 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:56.216954 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:56.217478 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:56.217867 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:56.217871 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:56.217874 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:56.218033 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:56.218696 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:56.218701 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:56.224929 (+ 6228us) client_negotiation.cc:769] Sending connection context
0809 15:07:56.225094 (+ 165us) client_negotiation.cc:240] Negotiation successful
0809 15:07:56.225400 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10855,"thread_start_us":10689,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.183.252@KRBTEST.COM" created.
I20260809 15:07:56.225912 3081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:56.192898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46818 (local address 127.24.183.253:39987)
0809 15:07:56.193242 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:56.193249 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:56.205042 (+ 11793us) server_negotiation.cc:407] Connection header received
0809 15:07:56.205253 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:56.205258 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:56.205359 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:56.205705 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:56.207030 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.208065 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.209847 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.210458 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.210544 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:56.213332 (+ 2788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:56.213337 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:56.213342 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:56.213374 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:56.216808 (+ 3434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:56.217596 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:56.217600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:56.217603 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:56.217747 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:56.218144 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:56.218149 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:56.218152 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:56.218491 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:56.218617 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:56.225229 (+ 6612us) server_negotiation.cc:299] Negotiation successful
0809 15:07:56.225557 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":111,"server-negotiator.queue_time_us":200}
Aug 09 15:07:56 dist-test-slave-nj21 krb5kdc[2156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288073, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.254@KRBTEST.COM for oryx/127.24.183.253@KRBTEST.COM
I20260809 15:07:56.255213 3081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:56.218955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46824 (local address 127.24.183.253:39987)
0809 15:07:56.227823 (+ 8868us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:56.227829 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:56.237698 (+ 9869us) server_negotiation.cc:407] Connection header received
0809 15:07:56.238124 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:56.238129 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:56.238253 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:56.238321 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:56.240001 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.241108 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.243138 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.243788 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.243887 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:56.248358 (+ 4471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:56.248365 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:56.248371 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:56.248410 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:56.252206 (+ 3796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:56.253099 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:56.253103 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:56.253105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:56.253241 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:56.253626 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:56.253629 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:56.253631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:56.253948 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:56.254060 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:56.254668 (+ 608us) server_negotiation.cc:299] Negotiation successful
0809 15:07:56.254855 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8763,"thread_start_us":16223,"threads_started":1}
I20260809 15:07:56.255275 3118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:56.218614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:39987 (local address 127.0.0.1:46824)
0809 15:07:56.237559 (+ 18945us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:56.237614 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:56.237865 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:56.238604 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:56.238612 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:56.239497 (+ 885us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:56.239815 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:56.239829 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.241356 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.241361 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:56.242983 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:56.242998 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.244235 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.244242 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:56.244435 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:56.245164 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:56.245186 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:56.248161 (+ 2975us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:56.252357 (+ 4196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:56.252370 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:56.252378 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:56.252954 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:56.253348 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:56.253353 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:56.253356 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:56.253525 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:56.254085 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:56.254093 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:56.254297 (+ 204us) client_negotiation.cc:769] Sending connection context
0809 15:07:56.254503 (+ 206us) client_negotiation.cc:240] Negotiation successful
0809 15:07:56.254834 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18546,"thread_start_us":18381,"threads_started":1}
I20260809 15:07:56.257297 2223 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.24.183.253:39987: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "d835c269415b4f52b71dcbac3af1a081"
member_type: NON_VOTER
last_known_addr {
host: "127.24.183.253"
port: 39987
}
attrs {
promote: true
}
}
dest_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81"
cas_config_opid_index: -1
I20260809 15:07:56.258334 2223 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: NON_VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: true } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.183.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.252.keytab.
Entry for principal oryx/127.24.183.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.252.keytab.
I20260809 15:07:56.261457 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.252:39741
--webserver_interface=127.24.183.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.252.keytab
--principal=oryx/127.24.183.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:37641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:45957,127.24.183.253:39987,127.24.183.252:39741
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:07:56.272660 3126 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER d835c269415b4f52b71dcbac3af1a081 (127.24.183.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: NON_VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: true } } }
I20260809 15:07:56.275481 3037 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:56.277010 3126 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.24.183.253:39987
I20260809 15:07:56.278255 2969 master.cc:537] Master@127.24.183.253:39987 shutting down...
I20260809 15:07:56.278980 3124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: NON_VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: true } } }
I20260809 15:07:56.279294 3124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:56.282121 3037 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [term 1 FOLLOWER]: Refusing update from remote peer 2be4818e2d7f4e51ba9ff391f3aecb81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 15:07:56.284381 3124 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: NON_VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 15:07:56.288151 2202 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260809 15:07:56.290535 2969 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:07:56.291116 2969 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:07:56.291404 2969 tablet_replica.cc:330] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081: stopping tablet replica
I20260809 15:07:56.299315 2969 master.cc:552] Master@127.24.183.253:39987 shutdown complete.
I20260809 15:07:56.299474 2969 master_runner.cc:305] Clearing existing system tablet
I20260809 15:07:56.300036 2969 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:56.300207 2969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
W20260809 15:07:56.301048 2202 consensus_peers.cc:487] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 -> Peer d835c269415b4f52b71dcbac3af1a081 (127.24.183.253:39987): Couldn't send request to peer d835c269415b4f52b71dcbac3af1a081. Status: Network error: recv got EOF from unknown peer (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260809 15:07:56.303160 2969 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:07:56.304648 3133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:56.304968 2969 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:56.305219 2969 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 15:07:56.305311 2969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-1/wal
uuid: "d835c269415b4f52b71dcbac3af1a081"
format_stamp: "Formatted at 2026-08-09 15:07:56 on dist-test-slave-nj21"
server_key: "ceff5dfa0dee62810316b9ca7f45332e"
server_key_iv: "2959be9e98d6da57f220cd5c31cc8d6e"
server_key_version: "encryptionkey@0"
I20260809 15:07:56.308691 2969 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:07:56.313602 2969 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 15:07:56.313767 2969 log.cc:1196] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 15:07:56.314141 2969 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081: Deleting consensus metadata
I20260809 15:07:56.314374 2969 master_runner.cc:315] Copying system tablet from 127.24.183.254:45957
I20260809 15:07:56.316615 2969 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081: tablet copy: Beginning tablet copy session from remote peer at address 127.24.183.254:45957
I20260809 15:07:56.328603 3082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:56.317286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37260 (local address 127.24.183.254:45957)
0809 15:07:56.317481 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:56.317488 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:56.317861 (+ 373us) server_negotiation.cc:407] Connection header received
0809 15:07:56.318062 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:56.318067 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:56.318138 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:56.318221 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:56.319110 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.319923 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.321132 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.321561 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.321669 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:56.323405 (+ 1736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:56.323417 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:56.323420 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:56.323452 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:56.326039 (+ 2587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:56.326631 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:56.326637 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:56.326639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:56.326768 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:56.327135 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:56.327140 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:56.327142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:56.327431 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:56.327568 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:56.328011 (+ 443us) server_negotiation.cc:299] Negotiation successful
0809 15:07:56.328213 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 15:07:56.328650 3140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:56.317264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.0.0.1:37260)
0809 15:07:56.317756 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:56.317776 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:56.317921 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:56.318260 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:56.318264 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:56.318717 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:56.318973 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:56.318986 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.320071 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.320076 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:56.320998 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:56.321010 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.321731 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.321737 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:56.321919 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:56.322493 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:56.322509 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:56.323216 (+ 707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:56.326165 (+ 2949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:56.326170 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:56.326175 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:56.326522 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:56.326873 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:56.326878 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:56.326882 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:56.327033 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:56.327549 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:56.327554 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:56.327751 (+ 197us) client_negotiation.cc:769] Sending connection context
0809 15:07:56.327897 (+ 146us) client_negotiation.cc:240] Negotiation successful
0809 15:07:56.328200 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":246,"threads_started":1}
I20260809 15:07:56.329612 2253 tablet_copy_service.cc:140] P 2be4818e2d7f4e51ba9ff391f3aecb81: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer d835c269415b4f52b71dcbac3af1a081 ({username='oryx', principal='oryx/127.24.183.253@KRBTEST.COM'} at 127.0.0.1:37260)
I20260809 15:07:56.329851 2253 tablet_copy_service.cc:161] P 2be4818e2d7f4e51ba9ff391f3aecb81: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer d835c269415b4f52b71dcbac3af1a081 at {username='oryx', principal='oryx/127.24.183.253@KRBTEST.COM'} at 127.0.0.1:37260: session id = d835c269415b4f52b71dcbac3af1a081-00000000000000000000000000000000
I20260809 15:07:56.332929 2253 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:07:56.334693 2969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 15:07:56.339473 2969 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081: tablet copy: Starting download of 0 data blocks...
I20260809 15:07:56.339779 2969 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081: tablet copy: Starting download of 1 WAL segments...
I20260809 15:07:56.342149 2969 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:07:56.345386 2253 tablet_copy_service.cc:342] P 2be4818e2d7f4e51ba9ff391f3aecb81: Request end of tablet copy session d835c269415b4f52b71dcbac3af1a081-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.24.183.253@KRBTEST.COM'} at 127.0.0.1:37260
I20260809 15:07:56.345554 2253 tablet_copy_service.cc:433] P 2be4818e2d7f4e51ba9ff391f3aecb81: ending tablet copy session d835c269415b4f52b71dcbac3af1a081-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer d835c269415b4f52b71dcbac3af1a081
I20260809 15:07:56.357910 2969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:56.360383 3147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:56.364233 3148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:56.364898 2969 server_base.cc:1034] running on GCE node
W20260809 15:07:56.365541 3151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:56.365993 2969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:56.366622 2969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:56.367787 2969 hybrid_clock.cc:648] HybridClock initialized: now 1786288076367712 us; error 34 us; skew 500 ppm
I20260809 15:07:56.368012 2969 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:56 dist-test-slave-nj21 krb5kdc[2156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288076, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:56.370217 2969 init.cc:377] Logged in from keytab as oryx/127.24.183.253@KRBTEST.COM (short username oryx)
I20260809 15:07:56.376649 2969 webserver.cc:460] Webserver started at http://127.24.183.253:37501/ using document root <none> and password file <none>
I20260809 15:07:56.377056 2969 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:56.377163 2969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:56.380515 2969 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.004s
I20260809 15:07:56.382540 3157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:56.382961 2969 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260809 15:07:56.383157 2969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-1/wal
uuid: "d835c269415b4f52b71dcbac3af1a081"
format_stamp: "Formatted at 2026-08-09 15:07:56 on dist-test-slave-nj21"
server_key: "ceff5dfa0dee62810316b9ca7f45332e"
server_key_iv: "2959be9e98d6da57f220cd5c31cc8d6e"
server_key_version: "encryptionkey@0"
I20260809 15:07:56.383397 2969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:56.411887 2969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:56.416119 2969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:56.436829 2969 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.253:39987
I20260809 15:07:56.436869 3235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.253:39987 every 8 connection(s)
I20260809 15:07:56.438009 2969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-1/data/info.pb
I20260809 15:07:56.444217 3236 sys_catalog.cc:263] Verifying existing consensus state
I20260809 15:07:56.447775 3236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081: Bootstrap starting.
I20260809 15:07:56.463572 3236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:07:56.464126 3236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081: Bootstrap complete.
I20260809 15:07:56.465186 3236 raft_consensus.cc:348] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: NON_VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: true } }
I20260809 15:07:56.465466 3236 raft_consensus.cc:729] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d835c269415b4f52b71dcbac3af1a081, State: Initialized, Role: LEARNER
I20260809 15:07:56.465739 3236 consensus_queue.cc:260] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: NON_VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: true } }
I20260809 15:07:56.466619 3241 sys_catalog.cc:455] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: NON_VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: true } } }
I20260809 15:07:56.466887 3236 sys_catalog.cc:564] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:07:56.467088 3241 sys_catalog.cc:458] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [sys.catalog]: This master's current role is: LEARNER
I20260809 15:07:56.475901 3255 catalog_manager.cc:1259] Loaded cluster ID: 8879a6c828534eb6bb5186bd3f176cb4
I20260809 15:07:56.475999 3255 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081: loading cluster ID for follower catalog manager: success
I20260809 15:07:56.479432 3255 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081: acquiring CA information for follower catalog manager: success
I20260809 15:07:56.481849 3255 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 15:07:56.625653 2962 heartbeater.cc:503] Master 127.24.183.254:45957 was elected leader, sending a full tablet report...
W20260809 15:07:56.658903 3127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:56.659214 3127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:56.659372 3127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:56.667070 3127 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:07:56.667173 3127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:56.667248 3127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:56.667330 3127 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:07:56.667390 3127 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:07:56.677489 3127 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:37641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:45957,127.24.183.253:39987,127.24.183.252:39741
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.252.keytab
--principal=oryx/127.24.183.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.252:39741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.183.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:56.678782 3127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:56.680091 3127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:56.691614 3264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:56.691543 3269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:56.691591 3266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:56.692420 3127 server_base.cc:1034] running on GCE node
I20260809 15:07:56.693117 3127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:56.694393 3127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:56.695611 3127 hybrid_clock.cc:648] HybridClock initialized: now 1786288076695576 us; error 25 us; skew 500 ppm
I20260809 15:07:56.695959 3127 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:56 dist-test-slave-nj21 krb5kdc[2156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288076, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:56.698923 3127 init.cc:377] Logged in from keytab as oryx/127.24.183.252@KRBTEST.COM (short username oryx)
I20260809 15:07:56.700343 3127 webserver.cc:460] Webserver started at http://127.24.183.252:38367/ using document root <none> and password file <none>
I20260809 15:07:56.701038 3127 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:56.701157 3127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:56.701453 3127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:07:56.704437 3127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-2/data/instance:
uuid: "eb2109a9820942409f99c6897b415642"
format_stamp: "Formatted at 2026-08-09 15:07:56 on dist-test-slave-nj21"
server_key: "99c7b8b48a741bd0cf7daae3db4a6e23"
server_key_iv: "7cbb03f0ab13eeafc2f1c235997924a9"
server_key_version: "encryptionkey@0"
I20260809 15:07:56.705216 3127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-2/wal/instance:
uuid: "eb2109a9820942409f99c6897b415642"
format_stamp: "Formatted at 2026-08-09 15:07:56 on dist-test-slave-nj21"
server_key: "99c7b8b48a741bd0cf7daae3db4a6e23"
server_key_iv: "7cbb03f0ab13eeafc2f1c235997924a9"
server_key_version: "encryptionkey@0"
I20260809 15:07:56.710132 3127 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 15:07:56.713497 3283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:56.715117 3127 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 15:07:56.715312 3127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-2/wal
uuid: "eb2109a9820942409f99c6897b415642"
format_stamp: "Formatted at 2026-08-09 15:07:56 on dist-test-slave-nj21"
server_key: "99c7b8b48a741bd0cf7daae3db4a6e23"
server_key_iv: "7cbb03f0ab13eeafc2f1c235997924a9"
server_key_version: "encryptionkey@0"
I20260809 15:07:56.715550 3127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:56.720242 3118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:56.713379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:39987 (local address 127.0.0.1:46826)
0809 15:07:56.713568 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:56.713587 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:56.713702 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:56.714620 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:56.714625 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:56.714655 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:07:56.714843 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:56.714855 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.716732 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.716737 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:56.717997 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:56.718011 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.719580 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.719584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:56.719807 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:56.719854 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 15:07:56.719931 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 15:07:56.720038 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 15:07:56.720453 3284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:56.713618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46826 (local address 127.24.183.253:39987)
0809 15:07:56.714226 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:56.714232 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:56.714257 (+ 25us) server_negotiation.cc:407] Connection header received
0809 15:07:56.714334 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:56.714339 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:56.714438 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:56.714558 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:07:56.714998 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.716586 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.718155 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.719436 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.719640 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:56.719854 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:56.719972 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 15:07:56.720107 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":310,"threads_started":1}
I20260809 15:07:56.721642 3200 raft_consensus.cc:1203] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260809 15:07:56.722314 3241 sys_catalog.cc:455] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2be4818e2d7f4e51ba9ff391f3aecb81. Latest consensus state: current_term: 1 leader_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: NON_VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: true } } }
I20260809 15:07:56.722551 3241 sys_catalog.cc:458] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [sys.catalog]: This master's current role is: LEARNER
I20260809 15:07:56.751955 3127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:56.755815 3127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:56.756078 3127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:56.773847 3127 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.252:39741
I20260809 15:07:56.773907 3381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.252:39741 every 8 connection(s)
I20260809 15:07:56.775149 3127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-2/data/info.pb
I20260809 15:07:56.775869 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 3127
I20260809 15:07:56.779349 3383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:56.787847 3383 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:07:56 dist-test-slave-nj21 krb5kdc[2156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.183.252@KRBTEST.COM
Aug 09 15:07:56 dist-test-slave-nj21 krb5kdc[2156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288076, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.252@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM
I20260809 15:07:56.799969 3392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:56.778084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39744 (local address 127.24.183.252:39741)
0809 15:07:56.778897 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:56.778908 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:56.778950 (+ 42us) server_negotiation.cc:407] Connection header received
0809 15:07:56.780666 (+ 1716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:56.780690 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:56.781039 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:56.781356 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:56.783157 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.784590 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.786407 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.787085 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.787204 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:56.792479 (+ 5275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:56.792492 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:56.792505 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:56.792552 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:56.796029 (+ 3477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:56.796645 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:56.796651 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:56.796657 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:56.796817 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:56.797171 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:56.797175 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:56.797177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:56.797777 (+ 600us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:56.798101 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:56.798563 (+ 462us) server_negotiation.cc:299] Negotiation successful
0809 15:07:56.798913 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":399,"threads_started":1}
I20260809 15:07:56.805565 3082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:56.790943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37266 (local address 127.24.183.254:45957)
0809 15:07:56.791367 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:56.791373 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:56.791632 (+ 259us) server_negotiation.cc:407] Connection header received
0809 15:07:56.791871 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:56.791876 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:56.791965 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:56.792072 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:56.793482 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.794278 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.796283 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.796725 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.796839 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:56.799987 (+ 3148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:56.799992 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:56.799995 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:56.800028 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:56.802866 (+ 2838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:56.803539 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:56.803545 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:56.803548 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:56.803721 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:56.804169 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:56.804175 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:56.804178 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:56.804457 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:56.804589 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:56.805038 (+ 449us) server_negotiation.cc:299] Negotiation successful
0809 15:07:56.805234 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295}
I20260809 15:07:56.805606 3393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:56.790773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.0.0.1:37266)
0809 15:07:56.791490 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:56.791529 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:56.791705 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:56.792128 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:56.792133 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:56.792991 (+ 858us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:56.793324 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:56.793336 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.794446 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.794451 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:56.796149 (+ 1698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:56.796161 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.796908 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.796915 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:56.797083 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:56.797613 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:56.797633 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:56.799810 (+ 2177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:56.802984 (+ 3174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:56.802995 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:56.803002 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:56.803418 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:56.803835 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:56.803840 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:56.803872 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:56.804055 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:56.804579 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:56.804586 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:56.804744 (+ 158us) client_negotiation.cc:769] Sending connection context
0809 15:07:56.804913 (+ 169us) client_negotiation.cc:240] Negotiation successful
0809 15:07:56.805201 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":363,"threads_started":1}
I20260809 15:07:56.806840 3383 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:07:56 dist-test-slave-nj21 krb5kdc[2156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288076, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.252@KRBTEST.COM for oryx/127.24.183.253@KRBTEST.COM
I20260809 15:07:56.818152 3393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:56.807435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:39987 (local address 127.0.0.1:46832)
0809 15:07:56.807627 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:56.807645 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:56.807796 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:56.808084 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:56.808086 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:56.808378 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:56.808514 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:56.808521 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.809395 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.809398 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:56.810292 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:56.810302 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.810906 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.810910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:56.811039 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:56.811485 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:56.811502 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:56.813401 (+ 1899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:56.816224 (+ 2823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:56.816229 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:56.816233 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:56.816626 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:56.816995 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:56.816998 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:56.817000 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:56.817126 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:56.817538 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:56.817541 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:56.817649 (+ 108us) client_negotiation.cc:769] Sending connection context
0809 15:07:56.817745 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 15:07:56.817939 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 15:07:56.818251 3284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:56.807642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46832 (local address 127.24.183.253:39987)
0809 15:07:56.807844 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:56.807849 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:56.807865 (+ 16us) server_negotiation.cc:407] Connection header received
0809 15:07:56.807924 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:56.807928 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:56.807991 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:56.808066 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:56.808615 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.809302 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.810400 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.810802 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.810889 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:56.813595 (+ 2706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:56.813599 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:56.813602 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:56.813634 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:56.816104 (+ 2470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:56.816744 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:56.816748 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:56.816750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:56.816902 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:56.817200 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:56.817203 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:56.817205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:56.817456 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:56.817568 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:56.817829 (+ 261us) server_negotiation.cc:299] Negotiation successful
0809 15:07:56.818002 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 15:07:56.819137 3383 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39741 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:07:56 dist-test-slave-nj21 krb5kdc[2156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288076, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.252@KRBTEST.COM for oryx/127.24.183.252@KRBTEST.COM
I20260809 15:07:56.830329 3393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:56.819898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:39741 (local address 127.0.0.1:39756)
0809 15:07:56.820052 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:56.820067 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:56.820148 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:56.820373 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:56.820376 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:56.820701 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:56.820875 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:56.820887 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.821803 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.821807 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:56.822645 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:56.822656 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.823322 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.823326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:56.823469 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:56.823945 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:56.823961 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:56.825791 (+ 1830us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:56.828294 (+ 2503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:56.828298 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:56.828300 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:56.828663 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:56.828957 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:56.828960 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:56.828963 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:56.829106 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:56.829622 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:56.829625 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:56.829749 (+ 124us) client_negotiation.cc:769] Sending connection context
0809 15:07:56.829855 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 15:07:56.830042 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 15:07:56.830504 3392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:56.819961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39756 (local address 127.24.183.252:39741)
0809 15:07:56.820125 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:56.820129 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:56.820144 (+ 15us) server_negotiation.cc:407] Connection header received
0809 15:07:56.820235 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:56.820239 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:56.820288 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:56.820355 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:56.820990 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.821674 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.822764 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.823191 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.823286 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:56.825943 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:56.825947 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:56.825951 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:56.825978 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:56.828182 (+ 2204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:56.828747 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:56.828751 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:56.828753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:56.828871 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:56.829242 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:56.829245 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:56.829247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:56.829527 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:56.829632 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:56.829936 (+ 304us) server_negotiation.cc:299] Negotiation successful
0809 15:07:56.830165 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":50}
I20260809 15:07:56.835134 3383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642: Bootstrap starting.
I20260809 15:07:56.838363 3383 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:56.839340 3383 log.cc:824] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642: Log is configured to *not* fsync() on all Append() calls
I20260809 15:07:56.842065 3383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642: No bootstrap required, opened a new log
I20260809 15:07:56.849082 3383 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39741 } }
I20260809 15:07:56.849377 3383 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:56.849502 3383 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb2109a9820942409f99c6897b415642, State: Initialized, Role: FOLLOWER
I20260809 15:07:56.850008 3383 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39741 } }
I20260809 15:07:56.850848 3395 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39741 } } }
I20260809 15:07:56.851157 3395 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:07:56.851802 3383 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 15:07:56.860383 3411 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:07:56.860524 3411 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:07:56.864349 3127 master_runner.cc:428] Detected that this master eb2109a9820942409f99c6897b415642 is joining an existing cluster
I20260809 15:07:56.864450 3127 master_runner.cc:432] Initiating AddMaster RPC to add 127.24.183.252:39741
I20260809 15:07:56.896404 3392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:56.870861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39764 (local address 127.24.183.252:39741)
0809 15:07:56.871048 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:56.871054 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:56.880735 (+ 9681us) server_negotiation.cc:407] Connection header received
0809 15:07:56.881170 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:56.881174 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:56.881237 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:56.881295 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:56.882937 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.883781 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.885605 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.886337 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.886427 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:56.889820 (+ 3393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:56.889827 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:56.889831 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:56.889864 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:56.893417 (+ 3553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:56.894180 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:56.894185 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:56.894188 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:56.894352 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:56.894751 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:56.894755 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:56.894758 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:56.895130 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:56.895266 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:56.895799 (+ 533us) server_negotiation.cc:299] Negotiation successful
0809 15:07:56.896027 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":52}
I20260809 15:07:56.896749 3082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:56.868141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37276 (local address 127.24.183.254:45957)
0809 15:07:56.868538 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:56.868545 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:56.878207 (+ 9662us) server_negotiation.cc:407] Connection header received
0809 15:07:56.878763 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:56.878770 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:56.878855 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:56.878933 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:56.880598 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.881749 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.884076 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.884757 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.884854 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:56.887903 (+ 3049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:56.887909 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:56.887914 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:56.887953 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:56.891445 (+ 3492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:56.893923 (+ 2478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:56.893929 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:56.893933 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:56.894098 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:56.894613 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:56.894618 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:56.894621 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:56.895016 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:56.895155 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:56.895862 (+ 707us) server_negotiation.cc:299] Negotiation successful
0809 15:07:56.896100 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265}
I20260809 15:07:56.896829 3284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:56.868324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46844 (local address 127.24.183.253:39987)
0809 15:07:56.868685 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:56.868691 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:56.880744 (+ 12053us) server_negotiation.cc:407] Connection header received
0809 15:07:56.881177 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:56.881182 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:56.881261 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:56.881324 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:56.883924 (+ 2600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.885024 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.887235 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.887890 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.887991 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:56.890744 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:56.890751 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:56.890755 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:56.890798 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:56.893821 (+ 3023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:56.894579 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:56.894584 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:56.894587 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:56.894738 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:56.895173 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:56.895178 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:56.895181 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:56.895525 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:56.895644 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:56.896270 (+ 626us) server_negotiation.cc:299] Negotiation successful
0809 15:07:56.896507 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243}
I20260809 15:07:56.897394 3424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:56.871425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:39741 (local address 127.0.0.1:39764)
0809 15:07:56.880661 (+ 9236us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:56.880675 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:56.881061 (+ 386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:56.881489 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:56.881494 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:56.882384 (+ 890us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:56.882762 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:56.882774 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.883959 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.883963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:56.885420 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:56.885443 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.887599 (+ 2156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.887606 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:56.887807 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:56.888424 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:56.888446 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:56.889650 (+ 1204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:56.893552 (+ 3902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:56.893558 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:56.893562 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:56.894072 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:56.894480 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:56.894484 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:56.894488 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:56.894649 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:56.895353 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:56.895357 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:56.895522 (+ 165us) client_negotiation.cc:769] Sending connection context
0809 15:07:56.896643 (+ 1121us) client_negotiation.cc:240] Negotiation successful
0809 15:07:56.896936 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9135,"spinlock_wait_cycles":20224,"thread_start_us":6800,"threads_started":1}
I20260809 15:07:56.897747 3425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:56.868321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:39987 (local address 127.0.0.1:46844)
0809 15:07:56.880630 (+ 12309us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:56.880651 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:56.881012 (+ 361us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:56.881503 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:56.881507 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:56.882613 (+ 1106us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:56.883739 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:56.883758 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.885262 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.885267 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:56.887067 (+ 1800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:56.887082 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.888287 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.888294 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:56.888579 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:56.889151 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:56.889175 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:56.890554 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:56.893970 (+ 3416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:56.893975 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:56.893979 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:56.894451 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:56.894879 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:56.894887 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:56.894890 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:56.895041 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:56.895778 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:56.895782 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:56.895975 (+ 193us) client_negotiation.cc:769] Sending connection context
0809 15:07:56.896989 (+ 1014us) client_negotiation.cc:240] Negotiation successful
0809 15:07:56.897286 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12175,"mutex_wait_us":62,"thread_start_us":12427,"threads_started":1}
I20260809 15:07:56.898831 3427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:56.868266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.0.0.1:37276)
0809 15:07:56.878098 (+ 9832us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:56.878123 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:56.878584 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:56.879163 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:56.879168 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:56.879899 (+ 731us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:56.880328 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:56.880399 (+ 71us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.882004 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.882009 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:56.883903 (+ 1894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:56.883919 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.885225 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.885232 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:56.885534 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:56.886409 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:56.886432 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:56.887698 (+ 1266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:56.891599 (+ 3901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:56.891604 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:56.891608 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:56.893788 (+ 2180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:56.894253 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:56.894258 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:56.894262 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:56.894476 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:56.895241 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:56.895245 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:56.895480 (+ 235us) client_negotiation.cc:769] Sending connection context
0809 15:07:56.897422 (+ 1942us) client_negotiation.cc:240] Negotiation successful
0809 15:07:56.898351 (+ 929us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9661,"spinlock_wait_cycles":896,"thread_start_us":12475,"threads_started":1}
I20260809 15:07:56.905483 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 2314
Aug 09 15:07:56 dist-test-slave-nj21 krb5kdc[2156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288073, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.254@KRBTEST.COM for oryx/127.24.183.252@KRBTEST.COM
I20260809 15:07:56.917975 3118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:56.903519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:39741 (local address 127.0.0.1:39780)
0809 15:07:56.903728 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:56.903747 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:56.903857 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:56.904306 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:56.904311 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:56.904741 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:56.904916 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:56.904927 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.906611 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.906617 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:56.907851 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:56.907864 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.909048 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.909054 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:56.909227 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:56.909827 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:56.909844 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:56.912118 (+ 2274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:56.915578 (+ 3460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:56.915584 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:56.915588 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:56.915986 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:56.916338 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:56.916344 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:56.916348 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:56.916477 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:56.916978 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:56.916983 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:56.917140 (+ 157us) client_negotiation.cc:769] Sending connection context
0809 15:07:56.917259 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 15:07:56.917579 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 15:07:56.917963 3392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:56.903580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39780 (local address 127.24.183.252:39741)
0809 15:07:56.903753 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:56.903758 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:56.903800 (+ 42us) server_negotiation.cc:407] Connection header received
0809 15:07:56.904063 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:56.904068 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:56.904175 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:56.904265 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:56.905225 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.906457 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.908033 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.908724 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.908857 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:56.912330 (+ 3473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:56.912337 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:56.912341 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:56.912382 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:56.915454 (+ 3072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:56.916106 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:56.916112 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:56.916114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:56.916232 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:56.916588 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:56.916591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:56.916593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:56.916862 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:56.916999 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:56.917415 (+ 416us) server_negotiation.cc:299] Negotiation successful
0809 15:07:56.917619 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
Aug 09 15:07:56 dist-test-slave-nj21 krb5kdc[2156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.183.193@KRBTEST.COM
I20260809 15:07:56.919834 2223 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.24.183.252:39741: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "eb2109a9820942409f99c6897b415642"
member_type: NON_VOTER
last_known_addr {
host: "127.24.183.252"
port: 39741
}
attrs {
promote: true
}
}
dest_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81"
cas_config_opid_index: 5
I20260809 15:07:56.921002 2223 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: NON_VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: true } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: NON_VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: true } }
I20260809 15:07:56.922844 3124 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER eb2109a9820942409f99c6897b415642 (127.24.183.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: NON_VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: true } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: NON_VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: true } } }
I20260809 15:07:56.923516 3200 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [term 1 LEARNER]: Refusing update from remote peer 2be4818e2d7f4e51ba9ff391f3aecb81: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 15:07:56.928100 3337 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:56.930015 3428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:56.906616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37832 (local address 127.24.183.193:44531)
0809 15:07:56.907312 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:56.907323 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:56.907358 (+ 35us) server_negotiation.cc:407] Connection header received
0809 15:07:56.907439 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:56.907448 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:56.907793 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:56.907963 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:56.909647 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.911287 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.913157 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:56.914023 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:56.914164 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:56.920097 (+ 5933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:56.920115 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:56.920136 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:56.920255 (+ 119us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:56.925643 (+ 5388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:56.926508 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:56.926517 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:56.926521 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:56.926692 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:56.927182 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:56.927187 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:56.927190 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:56.927648 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:56.927975 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:56.929293 (+ 1318us) server_negotiation.cc:299] Negotiation successful
0809 15:07:56.929595 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":574,"threads_started":1}
I20260809 15:07:56.931433 2521 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:07:56.931440 3429 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: NON_VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 15:07:56.932358 3337 raft_consensus.cc:1261] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [term 1 FOLLOWER]: Refusing update from remote peer 2be4818e2d7f4e51ba9ff391f3aecb81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 15:07:56.934229 3200 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER eb2109a9820942409f99c6897b415642 (127.24.183.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: NON_VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: true } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: NON_VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: true } } }
I20260809 15:07:56.938804 3241 sys_catalog.cc:455] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: NON_VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: true } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: NON_VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: true } } }
I20260809 15:07:56.939100 3241 sys_catalog.cc:458] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [sys.catalog]: This master's current role is: LEARNER
I20260809 15:07:56.941291 3124 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.24.183.252:39741
I20260809 15:07:56.941597 3124 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: NON_VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:07:56.942365 3433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: NON_VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: true } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: NON_VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: true } } }
I20260809 15:07:56.944056 3433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:56.944793 3430 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81: attempting to promote NON_VOTER d835c269415b4f52b71dcbac3af1a081 to VOTER
I20260809 15:07:56.949716 3127 master.cc:537] Master@127.24.183.252:39741 shutting down...
I20260809 15:07:56.949748 3430 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: false } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: NON_VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: true } }
I20260809 15:07:56.951859 3200 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [term 1 LEARNER]: Refusing update from remote peer 2be4818e2d7f4e51ba9ff391f3aecb81: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 15:07:56.952901 3430 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 15:07:56.956350 3124 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, d835c269415b4f52b71dcbac3af1a081 (127.24.183.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: false } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: NON_VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: true } } }
I20260809 15:07:56.956890 3200 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, d835c269415b4f52b71dcbac3af1a081 (127.24.183.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: false } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: NON_VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: true } } }
I20260809 15:07:56.960417 3437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: false } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: NON_VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: true } } }
I20260809 15:07:56.960857 3437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:56.961138 3241 sys_catalog.cc:455] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: false } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: NON_VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: true } } }
I20260809 15:07:56.961452 3241 sys_catalog.cc:458] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:07:56.961946 3437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: false } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: NON_VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: true } } }
I20260809 15:07:56.962235 3437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:56.971706 3337 raft_consensus.cc:858] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39741 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: NON_VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: NON_VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: true } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: NON_VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: true } }
W20260809 15:07:56.973101 2205 consensus_peers.cc:487] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 -> Peer eb2109a9820942409f99c6897b415642 (127.24.183.252:39741): Couldn't send request to peer eb2109a9820942409f99c6897b415642. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
W20260809 15:07:56.971972 3337 raft_consensus.cc:1565] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39741 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: NON_VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: NON_VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: true } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: NON_VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: true } }
I20260809 15:07:57.008558 3337 raft_consensus.cc:2932] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER d835c269415b4f52b71dcbac3af1a081 (127.24.183.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: NON_VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: true } } }
I20260809 15:07:57.013933 3449 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: NON_VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: true } } }
I20260809 15:07:57.014284 3449 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260809 15:07:57.014514 3395 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2be4818e2d7f4e51ba9ff391f3aecb81. Latest consensus state: current_term: 1 leader_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: NON_VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: true } } }
I20260809 15:07:57.014744 3395 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260809 15:07:57.013433 3453 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: NON_VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: true } } }
I20260809 15:07:57.015486 3453 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260809 15:07:57.016938 3127 raft_consensus.cc:2229] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260809 15:07:57.017408 3127 raft_consensus.cc:2258] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260809 15:07:57.017596 3127 tablet_replica.cc:330] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642: stopping tablet replica
I20260809 15:07:57.025684 3127 master.cc:552] Master@127.24.183.252:39741 shutdown complete.
I20260809 15:07:57.025768 3127 master_runner.cc:305] Clearing existing system tablet
I20260809 15:07:57.026171 3127 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:57.026301 3127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:57.029330 3127 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 15:07:57.030995 3456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:57.031407 3127 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:57.031863 3127 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260809 15:07:57.032021 3127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-2/wal
uuid: "eb2109a9820942409f99c6897b415642"
format_stamp: "Formatted at 2026-08-09 15:07:56 on dist-test-slave-nj21"
server_key: "99c7b8b48a741bd0cf7daae3db4a6e23"
server_key_iv: "7cbb03f0ab13eeafc2f1c235997924a9"
server_key_version: "encryptionkey@0"
I20260809 15:07:57.036695 3127 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:07:57.043766 3127 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 15:07:57.043999 3127 log.cc:1196] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 15:07:57.044590 3127 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642: Deleting consensus metadata
I20260809 15:07:57.044905 3127 master_runner.cc:315] Copying system tablet from 127.24.183.254:45957
I20260809 15:07:57.047847 3127 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642: tablet copy: Beginning tablet copy session from remote peer at address 127.24.183.254:45957
I20260809 15:07:57.067127 3082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:57.048797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37286 (local address 127.24.183.254:45957)
0809 15:07:57.049041 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:57.049048 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:57.052628 (+ 3580us) server_negotiation.cc:407] Connection header received
0809 15:07:57.052830 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:57.052835 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:57.052940 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:57.053059 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:57.054134 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:57.055325 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:57.057230 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:57.057893 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:57.058010 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:57.060364 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:57.060371 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:57.060376 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:57.060417 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:57.063938 (+ 3521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:57.064789 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:57.064795 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:57.064799 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:57.064982 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:57.065437 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:57.065442 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:57.065445 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:57.065890 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:57.066029 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:57.066504 (+ 475us) server_negotiation.cc:299] Negotiation successful
0809 15:07:57.066749 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260809 15:07:57.067631 3469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:57.048745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.0.0.1:37286)
0809 15:07:57.052501 (+ 3756us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:57.052526 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:57.052701 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:57.053110 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:57.053115 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:57.053687 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:57.053970 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:57.053984 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:57.055465 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:57.055470 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:57.057093 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:57.057109 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:57.058057 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:57.058064 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:57.058272 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:57.058997 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:57.059022 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:57.060145 (+ 1123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:57.064092 (+ 3947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:57.064098 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:57.064103 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:57.064657 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:57.065099 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:57.065103 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:57.065107 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:57.065314 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:57.066009 (+ 695us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:57.066014 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:57.066214 (+ 200us) client_negotiation.cc:769] Sending connection context
0809 15:07:57.066371 (+ 157us) client_negotiation.cc:240] Negotiation successful
0809 15:07:57.066709 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3574,"thread_start_us":3492,"threads_started":1}
I20260809 15:07:57.068317 2253 tablet_copy_service.cc:140] P 2be4818e2d7f4e51ba9ff391f3aecb81: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer eb2109a9820942409f99c6897b415642 ({username='oryx', principal='oryx/127.24.183.252@KRBTEST.COM'} at 127.0.0.1:37286)
I20260809 15:07:57.068495 2253 tablet_copy_service.cc:161] P 2be4818e2d7f4e51ba9ff391f3aecb81: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer eb2109a9820942409f99c6897b415642 at {username='oryx', principal='oryx/127.24.183.252@KRBTEST.COM'} at 127.0.0.1:37286: session id = eb2109a9820942409f99c6897b415642-00000000000000000000000000000000
I20260809 15:07:57.072049 2253 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:07:57.073975 3127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 15:07:57.077082 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 2314
I20260809 15:07:57.083662 3127 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642: tablet copy: Starting download of 0 data blocks...
I20260809 15:07:57.084061 3127 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642: tablet copy: Starting download of 1 WAL segments...
I20260809 15:07:57.086680 3127 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:07:57.091177 2253 tablet_copy_service.cc:342] P 2be4818e2d7f4e51ba9ff391f3aecb81: Request end of tablet copy session eb2109a9820942409f99c6897b415642-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.24.183.252@KRBTEST.COM'} at 127.0.0.1:37286
I20260809 15:07:57.091323 2253 tablet_copy_service.cc:433] P 2be4818e2d7f4e51ba9ff391f3aecb81: ending tablet copy session eb2109a9820942409f99c6897b415642-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer eb2109a9820942409f99c6897b415642
I20260809 15:07:57.102463 3127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:57.105127 3470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:57.109405 3471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:57.109983 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.193:44531
--local_ip_for_outbound_sockets=127.24.183.193
--tserver_master_addrs=127.24.183.254:45957
--webserver_port=38651
--webserver_interface=127.24.183.193
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.193.keytab
--principal=oryx/127.24.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:37641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.183.254:45957,127.24.183.253:39987
--tserver_master_addrs=127.24.183.254:45957,127.24.183.253:39987,127.24.183.252:39741 with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:07:57.115793 3474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:57.120322 3127 server_base.cc:1034] running on GCE node
I20260809 15:07:57.120661 3127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:57.121335 3127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:57.122550 3127 hybrid_clock.cc:648] HybridClock initialized: now 1786288077122497 us; error 33 us; skew 500 ppm
I20260809 15:07:57.122792 3127 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:57 dist-test-slave-nj21 krb5kdc[2156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288077, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:57.125066 3127 init.cc:377] Logged in from keytab as oryx/127.24.183.252@KRBTEST.COM (short username oryx)
I20260809 15:07:57.126359 3127 webserver.cc:460] Webserver started at http://127.24.183.252:43459/ using document root <none> and password file <none>
I20260809 15:07:57.126685 3127 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:57.126796 3127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:57.130765 3127 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:07:57.132695 3486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:57.133128 3127 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 15:07:57.133260 3127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-2/wal
uuid: "eb2109a9820942409f99c6897b415642"
format_stamp: "Formatted at 2026-08-09 15:07:56 on dist-test-slave-nj21"
server_key: "99c7b8b48a741bd0cf7daae3db4a6e23"
server_key_iv: "7cbb03f0ab13eeafc2f1c235997924a9"
server_key_version: "encryptionkey@0"
I20260809 15:07:57.133581 3127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:57.187832 3127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:57.192358 3127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:57.221671 3127 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.252:39741
I20260809 15:07:57.221755 3580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.252:39741 every 8 connection(s)
I20260809 15:07:57.222754 3127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/master-2/data/info.pb
I20260809 15:07:57.228199 3581 sys_catalog.cc:263] Verifying existing consensus state
I20260809 15:07:57.232283 3581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642: Bootstrap starting.
I20260809 15:07:57.249065 3581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:07:57.249617 3581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642: Bootstrap complete.
I20260809 15:07:57.250821 3581 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: false } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: NON_VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: true } }
I20260809 15:07:57.251147 3581 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb2109a9820942409f99c6897b415642, State: Initialized, Role: LEARNER
I20260809 15:07:57.251432 3581 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: false } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: NON_VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: true } }
I20260809 15:07:57.252440 3584 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: false } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: NON_VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: true } } }
I20260809 15:07:57.252764 3581 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:07:57.252978 3584 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [sys.catalog]: This master's current role is: LEARNER
I20260809 15:07:57.265575 3600 catalog_manager.cc:1259] Loaded cluster ID: 8879a6c828534eb6bb5186bd3f176cb4
I20260809 15:07:57.265687 3600 catalog_manager.cc:1552] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642: loading cluster ID for follower catalog manager: success
I20260809 15:07:57.269768 3600 catalog_manager.cc:1574] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642: acquiring CA information for follower catalog manager: success
I20260809 15:07:57.272377 3600 catalog_manager.cc:1602] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 15:07:57.483052 3478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:57.483353 3478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:57.483520 3478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:57.491190 3478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:57.491335 3478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:57.491542 3478 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.193
I20260809 15:07:57.497661 3615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:57.491903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:39741 (local address 127.0.0.1:39792)
0809 15:07:57.492340 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:57.492368 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:57.492514 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:57.492957 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:57.492963 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:57.492993 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:07:57.493171 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:57.493181 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:57.494429 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:57.494436 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:57.495759 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:57.495775 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:57.496911 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:57.496918 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:57.497087 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:57.497121 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 15:07:57.497212 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 15:07:57.497348 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":220,"threads_started":1}
I20260809 15:07:57.497689 3616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:57.492047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39792 (local address 127.24.183.252:39741)
0809 15:07:57.492655 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:57.492661 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:57.492679 (+ 18us) server_negotiation.cc:407] Connection header received
0809 15:07:57.492740 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:57.492744 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:57.492813 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:57.492915 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:07:57.493310 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:57.494281 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:57.495913 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:57.496734 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:57.496911 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:57.497103 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:57.497233 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 15:07:57.497376 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":343,"threads_started":1}
I20260809 15:07:57.498661 3539 raft_consensus.cc:1203] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260809 15:07:57.499112 3584 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2be4818e2d7f4e51ba9ff391f3aecb81. Latest consensus state: current_term: 1 leader_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: false } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: NON_VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: true } } }
I20260809 15:07:57.499387 3584 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [sys.catalog]: This master's current role is: LEARNER
I20260809 15:07:57.502211 3478 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:37641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.193.keytab
--principal=oryx/127.24.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.193:44531
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.183.193
--webserver_port=38651
--tserver_master_addrs=127.24.183.254:45957,127.24.183.253:39987,127.24.183.252:39741
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.193
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:57.503471 3478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:57.504839 3478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:57.518208 3624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:57.518285 3619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:57.518257 3621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:57.519268 3478 server_base.cc:1034] running on GCE node
I20260809 15:07:57.519949 3478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:57.520874 3478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:57.522117 3478 hybrid_clock.cc:648] HybridClock initialized: now 1786288077522057 us; error 41 us; skew 500 ppm
I20260809 15:07:57.522437 3478 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:57 dist-test-slave-nj21 krb5kdc[2156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288077, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:57.525509 3478 init.cc:377] Logged in from keytab as oryx/127.24.183.193@KRBTEST.COM (short username oryx)
I20260809 15:07:57.526883 3478 webserver.cc:460] Webserver started at http://127.24.183.193:38651/ using document root <none> and password file <none>
I20260809 15:07:57.527513 3478 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:57.527626 3478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:57.532828 3478 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260809 15:07:57.535506 3634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:57.536849 3478 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:07:57.536995 3478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-0/wal
uuid: "cfc40b3492174e7cb2dfd14d21e72582"
format_stamp: "Formatted at 2026-08-09 15:07:54 on dist-test-slave-nj21"
server_key: "62283c587478ca97c94d378438535804"
server_key_iv: "04e5e1f2323291845b82c36c2c03437e"
server_key_version: "encryptionkey@0"
I20260809 15:07:57.537932 3478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:57.579126 3478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:57.582926 3478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:57.583155 3478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:57.584105 3478 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:07:57.585445 3478 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:07:57.585534 3478 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:57.585610 3478 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:07:57.585664 3478 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:57.617053 3478 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.193:44531
I20260809 15:07:57.617133 3826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.193:44531 every 8 connection(s)
I20260809 15:07:57.618444 3478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
I20260809 15:07:57.627359 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 3478
I20260809 15:07:57.627821 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 2538
Aug 09 15:07:57 dist-test-slave-nj21 krb5kdc[2156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288077, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.253@KRBTEST.COM
Aug 09 15:07:57 dist-test-slave-nj21 krb5kdc[2156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288077, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM
Aug 09 15:07:57 dist-test-slave-nj21 krb5kdc[2156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288077, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.252@KRBTEST.COM
I20260809 15:07:57.644709 3840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:57.621933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:48127 (local address 127.24.183.253:39987)
0809 15:07:57.622473 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:57.622480 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:57.623143 (+ 663us) server_negotiation.cc:407] Connection header received
0809 15:07:57.625720 (+ 2577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:57.625726 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:57.625821 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:57.625945 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:57.628417 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:57.629612 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:57.632060 (+ 2448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:57.632705 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:57.632801 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:57.637416 (+ 4615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:57.637422 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:57.637427 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:57.637466 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:57.640841 (+ 3375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:57.641807 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:57.641812 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:57.641815 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:57.641995 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:57.642625 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:57.642631 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:57.642634 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:57.643002 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:57.643148 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:57.644114 (+ 966us) server_negotiation.cc:299] Negotiation successful
0809 15:07:57.644323 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":275,"threads_started":1}
I20260809 15:07:57.647054 3843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:57.621789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:39987 (local address 127.24.183.193:48127)
0809 15:07:57.622971 (+ 1182us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:57.623024 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:57.625005 (+ 1981us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:57.625999 (+ 994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:57.626004 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:57.626705 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:57.628206 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:57.628241 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:57.629824 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:57.629828 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:57.631872 (+ 2044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:57.631887 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:57.633174 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:57.633182 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:57.633460 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:57.634234 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:57.634263 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:57.637172 (+ 2909us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:57.641049 (+ 3877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:57.641057 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:57.641071 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:57.641676 (+ 605us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:57.642219 (+ 543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:57.642223 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:57.642227 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:57.642490 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:57.643246 (+ 756us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:57.643253 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:57.643659 (+ 406us) client_negotiation.cc:769] Sending connection context
0809 15:07:57.644997 (+ 1338us) client_negotiation.cc:240] Negotiation successful
0809 15:07:57.645533 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":848,"mutex_wait_us":84,"spinlock_wait_cycles":87424,"thread_start_us":1680,"threads_started":1}
I20260809 15:07:57.648257 3837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:57.621202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:46841 (local address 127.24.183.254:45957)
0809 15:07:57.621766 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:57.621772 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:57.623136 (+ 1364us) server_negotiation.cc:407] Connection header received
0809 15:07:57.625267 (+ 2131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:57.625273 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:57.625354 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:57.625454 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:57.629821 (+ 4367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:57.630937 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:57.636364 (+ 5427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:57.637024 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:57.637127 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:57.641011 (+ 3884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:57.641018 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:57.641022 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:57.641063 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:57.644802 (+ 3739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:57.645699 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:57.645704 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:57.645707 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:57.645855 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:57.646292 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:57.646296 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:57.646299 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:57.646639 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:57.646768 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:57.647291 (+ 523us) server_negotiation.cc:299] Negotiation successful
0809 15:07:57.647877 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":316,"threads_started":1}
I20260809 15:07:57.648728 3835 heartbeater.cc:347] Connected to a master server at 127.24.183.253:39987
I20260809 15:07:57.649047 3835 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:57.649183 3847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:57.621964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.24.183.193:46841)
0809 15:07:57.622971 (+ 1007us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:57.623023 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:57.625006 (+ 1983us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:57.625698 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:57.625705 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:57.626705 (+ 1000us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:57.628374 (+ 1669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:57.628386 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:57.631284 (+ 2898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:57.631288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:57.634234 (+ 2946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:57.634248 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:57.637424 (+ 3176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:57.637431 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:57.637587 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:57.638128 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:57.638151 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:57.640797 (+ 2646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:57.644968 (+ 4171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:57.644973 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:57.644976 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:57.645567 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:57.645994 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:57.645999 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:57.646003 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:57.646160 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:57.646850 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:57.646854 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:57.647009 (+ 155us) client_negotiation.cc:769] Sending connection context
0809 15:07:57.648503 (+ 1494us) client_negotiation.cc:240] Negotiation successful
0809 15:07:57.648779 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":879,"mutex_wait_us":545,"thread_start_us":541,"threads_started":1}
I20260809 15:07:57.649386 3616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:57.621187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:60679 (local address 127.24.183.252:39741)
0809 15:07:57.621402 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:57.621408 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:57.623973 (+ 2565us) server_negotiation.cc:407] Connection header received
0809 15:07:57.625366 (+ 1393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:57.625371 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:57.625440 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:57.625522 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:57.628553 (+ 3031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:57.629653 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:57.634434 (+ 4781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:57.635067 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:57.635172 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:57.641986 (+ 6814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:57.641992 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:57.641996 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:57.642038 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:57.645383 (+ 3345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:57.646581 (+ 1198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:57.646587 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:57.646590 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:57.646746 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:57.647468 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:57.647473 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:57.647477 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:57.647902 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:57.648041 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:57.648796 (+ 755us) server_negotiation.cc:299] Negotiation successful
0809 15:07:57.649044 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 15:07:57.650018 3835 heartbeater.cc:511] Master 127.24.183.253:39987 requested a full tablet report, sending...
I20260809 15:07:57.650041 3828 heartbeater.cc:347] Connected to a master server at 127.24.183.254:45957
I20260809 15:07:57.650197 3828 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:57.650753 3828 heartbeater.cc:511] Master 127.24.183.254:45957 requested a full tablet report, sending...
I20260809 15:07:57.650959 3845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:57.622213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:39741 (local address 127.24.183.193:60679)
0809 15:07:57.623872 (+ 1659us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:57.623886 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:57.625011 (+ 1125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:57.625701 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:57.625708 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:57.626698 (+ 990us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:57.628206 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:57.628241 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:57.631126 (+ 2885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:57.631131 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:57.633055 (+ 1924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:57.633066 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:57.635502 (+ 2436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:57.635508 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:57.635723 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:57.639028 (+ 3305us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:57.639054 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:57.641739 (+ 2685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:57.645777 (+ 4038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:57.645782 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:57.645786 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:57.646447 (+ 661us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:57.646901 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:57.646905 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:57.646908 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:57.647078 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:57.648130 (+ 1052us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:57.648135 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:57.648279 (+ 144us) client_negotiation.cc:769] Sending connection context
0809 15:07:57.650010 (+ 1731us) client_negotiation.cc:240] Negotiation successful
0809 15:07:57.650398 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1555,"mutex_wait_us":691,"spinlock_wait_cycles":28928,"thread_start_us":1078,"threads_started":1}
Aug 09 15:07:57 dist-test-slave-nj21 krb5kdc[2156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.183.194@KRBTEST.COM
I20260809 15:07:57.651917 2223 ts_manager.cc:194] Re-registered known tserver with Master: cfc40b3492174e7cb2dfd14d21e72582 (127.24.183.193:44531)
I20260809 15:07:57.652274 3827 heartbeater.cc:347] Connected to a master server at 127.24.183.252:39741
I20260809 15:07:57.652396 3827 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:57.652468 3186 ts_manager.cc:194] Registered new tserver with Master: cfc40b3492174e7cb2dfd14d21e72582 (127.24.183.193:44531)
I20260809 15:07:57.652812 3827 heartbeater.cc:511] Master 127.24.183.252:39741 requested a full tablet report, sending...
I20260809 15:07:57.652974 2223 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.183.193@KRBTEST.COM'} at 127.24.183.193:46841
I20260809 15:07:57.654706 3514 ts_manager.cc:194] Registered new tserver with Master: cfc40b3492174e7cb2dfd14d21e72582 (127.24.183.193:44531)
I20260809 15:07:57.658699 3849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:57.629699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48560 (local address 127.24.183.194:46409)
0809 15:07:57.638622 (+ 8923us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:57.638631 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:57.639585 (+ 954us) server_negotiation.cc:407] Connection header received
0809 15:07:57.639665 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:57.639694 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:57.639940 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:57.642339 (+ 2399us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:57.642940 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:57.644482 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:57.646250 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:57.647059 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:57.647176 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:57.652946 (+ 5770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:57.652960 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:57.652974 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:57.653014 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:57.656329 (+ 3315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:57.656985 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:57.656990 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:57.656993 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:57.657125 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:57.657429 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:57.657432 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:57.657434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:57.657734 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:57.657954 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:57.658167 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 15:07:57.658346 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8753,"thread_start_us":8595,"threads_started":1}
I20260809 15:07:57.659845 2739 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:07:57.748063 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 2538
I20260809 15:07:57.772487 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.194:46409
--local_ip_for_outbound_sockets=127.24.183.194
--tserver_master_addrs=127.24.183.254:45957
--webserver_port=43313
--webserver_interface=127.24.183.194
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.194.keytab
--principal=oryx/127.24.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:37641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.183.254:45957,127.24.183.253:39987
--tserver_master_addrs=127.24.183.254:45957,127.24.183.253:39987,127.24.183.252:39741 with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:07:57.909777 3433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: false } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: NON_VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: true } } }
I20260809 15:07:57.910140 3433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:57.912001 3857 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81: attempting to promote NON_VOTER eb2109a9820942409f99c6897b415642 to VOTER
I20260809 15:07:57.912628 3857 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: false } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: false } }
I20260809 15:07:57.914559 3539 raft_consensus.cc:1261] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [term 1 LEARNER]: Refusing update from remote peer 2be4818e2d7f4e51ba9ff391f3aecb81: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 15:07:57.914922 3200 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [term 1 FOLLOWER]: Refusing update from remote peer 2be4818e2d7f4e51ba9ff391f3aecb81: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 15:07:57.915179 3433 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 15:07:57.915719 3858 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 15:07:57.918550 3433 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, eb2109a9820942409f99c6897b415642 (127.24.183.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: false } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: false } } }
I20260809 15:07:57.919052 3200 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, eb2109a9820942409f99c6897b415642 (127.24.183.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: false } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: false } } }
I20260809 15:07:57.921603 3433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: false } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: false } } }
I20260809 15:07:57.921805 3433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:57.922132 3433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: false } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: false } } }
I20260809 15:07:57.922398 3433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:57.922654 3858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: false } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: false } } }
I20260809 15:07:57.922927 3858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81 [sys.catalog]: This master's current role is: LEADER
I20260809 15:07:57.922608 3539 raft_consensus.cc:2932] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, eb2109a9820942409f99c6897b415642 (127.24.183.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: false } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: false } } }
I20260809 15:07:57.923928 3862 sys_catalog.cc:455] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: false } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: false } } }
I20260809 15:07:57.924343 3862 sys_catalog.cc:458] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:07:57.925967 3584 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: false } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: false } } }
I20260809 15:07:57.926179 3584 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 15:07:58.106848 3856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:58.107134 3856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:58.107280 3856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:58.114548 3856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:58.114660 3856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:58.114809 3856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.194
I20260809 15:07:58.124248 3856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:37641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.194.keytab
--principal=oryx/127.24.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.194:46409
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.183.194
--webserver_port=43313
--tserver_master_addrs=127.24.183.254:45957,127.24.183.253:39987,127.24.183.252:39741
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.194
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:58.125479 3856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:58.126732 3856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:58.139719 3873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:58.139644 3876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:58.139818 3868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:58.140645 3856 server_base.cc:1034] running on GCE node
I20260809 15:07:58.141270 3856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:58.142194 3856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:58.143406 3856 hybrid_clock.cc:648] HybridClock initialized: now 1786288078143326 us; error 65 us; skew 500 ppm
I20260809 15:07:58.143759 3856 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:58 dist-test-slave-nj21 krb5kdc[2156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288078, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:58.146809 3856 init.cc:377] Logged in from keytab as oryx/127.24.183.194@KRBTEST.COM (short username oryx)
I20260809 15:07:58.148213 3856 webserver.cc:460] Webserver started at http://127.24.183.194:43313/ using document root <none> and password file <none>
I20260809 15:07:58.148880 3856 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:58.149015 3856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:58.154633 3856 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 15:07:58.157410 3888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:58.158731 3856 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 15:07:58.158882 3856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-1/wal
uuid: "10025c6198174f61b5495c94ea3a510e"
format_stamp: "Formatted at 2026-08-09 15:07:55 on dist-test-slave-nj21"
server_key: "f6862c29cd973827d277b573997f04e7"
server_key_iv: "8ce231dc9f80243f757bbd8858e5dd23"
server_key_version: "encryptionkey@0"
I20260809 15:07:58.159901 3856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:58.176429 3856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:58.180456 3856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:58.180706 3856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:58.181680 3856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:07:58.183012 3856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:07:58.183107 3856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:58.183212 3856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:07:58.183283 3856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:58.213748 3856 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.194:46409
I20260809 15:07:58.213815 4081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.194:46409 every 8 connection(s)
I20260809 15:07:58.215176 3856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
I20260809 15:07:58.224182 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 3856
I20260809 15:07:58.224503 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 2761
Aug 09 15:07:58 dist-test-slave-nj21 krb5kdc[2156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288078, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.252@KRBTEST.COM
I20260809 15:07:58.269302 4096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:58.219400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:57913 (local address 127.24.183.252:39741)
0809 15:07:58.244264 (+ 24864us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:58.244272 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:58.248295 (+ 4023us) server_negotiation.cc:407] Connection header received
0809 15:07:58.248393 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:58.248399 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:58.248494 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:58.248600 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:58.251160 (+ 2560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.252543 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.255323 (+ 2780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.256042 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.256143 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:58.261493 (+ 5350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:58.261500 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:58.261506 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:58.261555 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:58.265232 (+ 3677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:58.266245 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:58.266251 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:58.266254 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:58.266406 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:58.267057 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:58.267063 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:58.267066 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:58.267435 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:58.267555 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:58.268651 (+ 1096us) server_negotiation.cc:299] Negotiation successful
0809 15:07:58.268895 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24662,"thread_start_us":24489,"threads_started":1}
Aug 09 15:07:58 dist-test-slave-nj21 krb5kdc[2156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.183.195@KRBTEST.COM
I20260809 15:07:58.271104 4095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:58.218692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:39741 (local address 127.24.183.194:57913)
0809 15:07:58.244952 (+ 26260us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:58.245021 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:58.247893 (+ 2872us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:58.249143 (+ 1250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:58.249156 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:58.250012 (+ 856us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:58.250937 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:58.250964 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.252829 (+ 1865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.252835 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:58.255139 (+ 2304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:58.255156 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.256581 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.256588 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:58.256904 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:58.257729 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:58.257772 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:58.261177 (+ 3405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:58.265443 (+ 4266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:58.265451 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:58.265468 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:58.266079 (+ 611us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:58.266564 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:58.266568 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:58.266572 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:58.266918 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:58.267659 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:58.267667 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:58.268204 (+ 537us) client_negotiation.cc:769] Sending connection context
0809 15:07:58.269147 (+ 943us) client_negotiation.cc:240] Negotiation successful
0809 15:07:58.269787 (+ 640us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25892,"thread_start_us":25634,"threads_started":1}
Aug 09 15:07:58 dist-test-slave-nj21 krb5kdc[2156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288078, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM
Aug 09 15:07:58 dist-test-slave-nj21 krb5kdc[2156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288078, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.253@KRBTEST.COM
I20260809 15:07:58.273128 4082 heartbeater.cc:347] Connected to a master server at 127.24.183.252:39741
I20260809 15:07:58.273414 4082 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:58.274327 4082 heartbeater.cc:511] Master 127.24.183.252:39741 requested a full tablet report, sending...
I20260809 15:07:58.276266 3514 ts_manager.cc:194] Registered new tserver with Master: 10025c6198174f61b5495c94ea3a510e (127.24.183.194:46409)
I20260809 15:07:58.278827 4103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:58.225858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36424 (local address 127.24.183.195:40011)
0809 15:07:58.254260 (+ 28402us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:58.254268 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:58.254684 (+ 416us) server_negotiation.cc:407] Connection header received
0809 15:07:58.254769 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:58.254778 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:58.255054 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:58.255203 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:58.257927 (+ 2724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.260507 (+ 2580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.262600 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.264327 (+ 1727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.264463 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:58.271188 (+ 6725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:58.271204 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:58.271221 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:58.271264 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:58.275495 (+ 4231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:58.276201 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:58.276209 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:58.276212 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:58.276377 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:58.276829 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:58.276834 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:58.276837 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:58.277272 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:58.277569 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:58.278186 (+ 617us) server_negotiation.cc:299] Negotiation successful
0809 15:07:58.278457 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28205,"thread_start_us":28622,"threads_started":1}
I20260809 15:07:58.280165 2956 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:07:58.281090 4090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:58.218011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:58733 (local address 127.24.183.254:45957)
0809 15:07:58.263232 (+ 45221us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:58.263239 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:58.263259 (+ 20us) server_negotiation.cc:407] Connection header received
0809 15:07:58.263323 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:58.263329 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:58.263423 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:58.263799 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:58.265355 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.266392 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.268423 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.269055 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.269885 (+ 830us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:58.273674 (+ 3789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:58.273681 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:58.273685 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:58.273727 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:58.277742 (+ 4015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:58.279051 (+ 1309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:58.279056 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:58.279060 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:58.279222 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:58.279657 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:58.279662 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:58.279665 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:58.279975 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:58.280106 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:58.280529 (+ 423us) server_negotiation.cc:299] Negotiation successful
0809 15:07:58.280734 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45084,"thread_start_us":44971,"threads_started":1}
I20260809 15:07:58.281162 4099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:58.219864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.24.183.194:58733)
0809 15:07:58.253020 (+ 33156us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:58.253043 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:58.253191 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:58.264194 (+ 11003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:58.264199 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:58.264872 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:58.265173 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:58.265189 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.266565 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.266571 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:58.268230 (+ 1659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:58.268245 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.269318 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.269327 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:58.269539 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:58.270157 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:58.270179 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:58.273454 (+ 3275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:58.277869 (+ 4415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:58.277874 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:58.277878 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:58.278382 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:58.279344 (+ 962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:58.279349 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:58.279352 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:58.279542 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:58.280093 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:58.280098 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:58.280244 (+ 146us) client_negotiation.cc:769] Sending connection context
0809 15:07:58.280381 (+ 137us) client_negotiation.cc:240] Negotiation successful
0809 15:07:58.280661 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32996,"spinlock_wait_cycles":25344,"thread_start_us":32760,"threads_started":1}
I20260809 15:07:58.282117 4085 heartbeater.cc:347] Connected to a master server at 127.24.183.254:45957
I20260809 15:07:58.282258 4094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:58.218072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:52133 (local address 127.24.183.253:39987)
0809 15:07:58.262977 (+ 44905us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:58.262983 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:58.263005 (+ 22us) server_negotiation.cc:407] Connection header received
0809 15:07:58.263065 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:58.263070 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:58.263159 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:58.263257 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:58.264759 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.265827 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.267751 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.268329 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.268450 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:58.274643 (+ 6193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:58.274649 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:58.274653 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:58.274709 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:58.278408 (+ 3699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:58.279064 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:58.279069 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:58.279072 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:58.279221 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:58.279651 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:58.279655 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:58.279658 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:58.280094 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:58.280244 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:58.281359 (+ 1115us) server_negotiation.cc:299] Negotiation successful
0809 15:07:58.281656 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44752,"thread_start_us":44619,"threads_started":1}
I20260809 15:07:58.282271 4085 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:58.282699 4100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:58.220326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:39987 (local address 127.24.183.194:52133)
0809 15:07:58.253759 (+ 33433us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:58.253783 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:58.253898 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:58.263313 (+ 9415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:58.263318 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:58.264115 (+ 797us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:58.264504 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:58.264570 (+ 66us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.266008 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.266013 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:58.267531 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:58.267545 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.268697 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.268704 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:58.268863 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:58.271247 (+ 2384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:58.271271 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:58.274336 (+ 3065us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:58.278542 (+ 4206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:58.278546 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:58.278550 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:58.278928 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:58.279340 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:58.279345 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:58.279349 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:58.279515 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:58.280324 (+ 809us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:58.280329 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:58.280506 (+ 177us) client_negotiation.cc:769] Sending connection context
0809 15:07:58.280645 (+ 139us) client_negotiation.cc:240] Negotiation successful
0809 15:07:58.280999 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33318,"thread_start_us":33178,"threads_started":1}
I20260809 15:07:58.282819 4085 heartbeater.cc:511] Master 127.24.183.254:45957 requested a full tablet report, sending...
I20260809 15:07:58.283609 4087 heartbeater.cc:347] Connected to a master server at 127.24.183.253:39987
I20260809 15:07:58.283761 4087 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:58.283927 2223 ts_manager.cc:194] Re-registered known tserver with Master: 10025c6198174f61b5495c94ea3a510e (127.24.183.194:46409)
I20260809 15:07:58.284195 4087 heartbeater.cc:511] Master 127.24.183.253:39987 requested a full tablet report, sending...
I20260809 15:07:58.285008 2223 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.183.194@KRBTEST.COM'} at 127.24.183.194:58733
I20260809 15:07:58.285430 3186 ts_manager.cc:194] Registered new tserver with Master: 10025c6198174f61b5495c94ea3a510e (127.24.183.194:46409)
I20260809 15:07:58.376199 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 2761
I20260809 15:07:58.399300 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.195:40011
--local_ip_for_outbound_sockets=127.24.183.195
--tserver_master_addrs=127.24.183.254:45957
--webserver_port=36527
--webserver_interface=127.24.183.195
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.195.keytab
--principal=oryx/127.24.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:37641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.183.254:45957,127.24.183.253:39987
--tserver_master_addrs=127.24.183.254:45957,127.24.183.253:39987,127.24.183.252:39741 with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:07:58.655205 3828 heartbeater.cc:503] Master 127.24.183.254:45957 was elected leader, sending a full tablet report...
W20260809 15:07:58.732050 4105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:07:58.732306 4105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:07:58.732402 4105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:07:58.739529 4105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:07:58.739632 4105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:07:58.739782 4105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.195
I20260809 15:07:58.749243 4105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:37641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.195.keytab
--principal=oryx/127.24.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.195:40011
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.183.195
--webserver_port=36527
--tserver_master_addrs=127.24.183.254:45957,127.24.183.253:39987,127.24.183.252:39741
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.195
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:07:58.750475 4105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:07:58.751806 4105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:07:58.764914 4112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:58.764993 4113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:07:58.765188 4115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:07:58.765882 4105 server_base.cc:1034] running on GCE node
I20260809 15:07:58.766438 4105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:07:58.767341 4105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:07:58.768533 4105 hybrid_clock.cc:648] HybridClock initialized: now 1786288078768479 us; error 45 us; skew 500 ppm
I20260809 15:07:58.768839 4105 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:07:58 dist-test-slave-nj21 krb5kdc[2156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288078, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:07:58.772043 4105 init.cc:377] Logged in from keytab as oryx/127.24.183.195@KRBTEST.COM (short username oryx)
I20260809 15:07:58.773438 4105 webserver.cc:460] Webserver started at http://127.24.183.195:36527/ using document root <none> and password file <none>
I20260809 15:07:58.774116 4105 fs_manager.cc:359] Metadata directory not provided
I20260809 15:07:58.774250 4105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:07:58.779757 4105 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 15:07:58.782429 4127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:58.783793 4105 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 15:07:58.783950 4105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-2/wal
uuid: "2ed67a7ab72e4760917f53986f1a774a"
format_stamp: "Formatted at 2026-08-09 15:07:55 on dist-test-slave-nj21"
server_key: "f313318271435ef0f83e6396b1356c66"
server_key_iv: "005b5473dc8db234a59e8e4893770ef1"
server_key_version: "encryptionkey@0"
I20260809 15:07:58.784961 4105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:07:58.808657 4105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:07:58.812353 4105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:07:58.812621 4105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:07:58.813583 4105 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:07:58.814904 4105 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:07:58.815009 4105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:58.815124 4105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:07:58.815193 4105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:07:58.845432 4105 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.195:40011
I20260809 15:07:58.845499 4299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.195:40011 every 8 connection(s)
I20260809 15:07:58.846838 4105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
I20260809 15:07:58.851045 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 4105
Aug 09 15:07:58 dist-test-slave-nj21 krb5kdc[2156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288078, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.253@KRBTEST.COM
Aug 09 15:07:58 dist-test-slave-nj21 krb5kdc[2156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288078, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.252@KRBTEST.COM
Aug 09 15:07:58 dist-test-slave-nj21 krb5kdc[2156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288078, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM
I20260809 15:07:58.887435 4322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:58.858378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:33611 (local address 127.24.183.253:39987)
0809 15:07:58.861245 (+ 2867us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:58.861252 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:58.863195 (+ 1943us) server_negotiation.cc:407] Connection header received
0809 15:07:58.863427 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:58.863433 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:58.863537 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:58.863642 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:58.866378 (+ 2736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.867525 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.871947 (+ 4422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.872363 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.872485 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:58.877638 (+ 5153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:58.877645 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:58.877650 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:58.877694 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:58.882133 (+ 4439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:58.883899 (+ 1766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:58.883906 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:58.883910 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:58.884083 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:58.885109 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:58.885115 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:58.885119 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:58.885673 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:58.885812 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:58.886813 (+ 1001us) server_negotiation.cc:299] Negotiation successful
0809 15:07:58.887044 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2727,"thread_start_us":2616,"threads_started":1}
I20260809 15:07:58.888703 4315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:58.857823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:39987 (local address 127.24.183.195:33611)
0809 15:07:58.863068 (+ 5245us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:58.863087 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:58.863300 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:58.864292 (+ 992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:58.864298 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:58.865102 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:58.866075 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:58.866092 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.867695 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.867701 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:58.869978 (+ 2277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:58.869995 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.872587 (+ 2592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.872595 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:58.872886 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:58.873677 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:58.873707 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:58.877408 (+ 3701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:58.882255 (+ 4847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:58.882268 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:58.882284 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:58.882941 (+ 657us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:58.884226 (+ 1285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:58.884232 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:58.884236 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:58.884972 (+ 736us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:58.885940 (+ 968us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:58.885948 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:58.886429 (+ 481us) client_negotiation.cc:769] Sending connection context
0809 15:07:58.886733 (+ 304us) client_negotiation.cc:240] Negotiation successful
0809 15:07:58.887220 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5117,"mutex_wait_us":149,"spinlock_wait_cycles":25344,"thread_start_us":2531,"threads_started":1}
I20260809 15:07:58.890511 4306 heartbeater.cc:347] Connected to a master server at 127.24.183.253:39987
I20260809 15:07:58.890870 4306 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:58.891944 4306 heartbeater.cc:511] Master 127.24.183.253:39987 requested a full tablet report, sending...
I20260809 15:07:58.894737 3186 ts_manager.cc:194] Registered new tserver with Master: 2ed67a7ab72e4760917f53986f1a774a (127.24.183.195:40011)
I20260809 15:07:58.899500 4311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:58.850782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:42593 (local address 127.24.183.254:45957)
0809 15:07:58.863774 (+ 12992us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:58.863782 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:58.863802 (+ 20us) server_negotiation.cc:407] Connection header received
0809 15:07:58.863882 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:58.863887 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:58.863969 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:58.864084 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:58.866351 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.867524 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.875934 (+ 8410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.876377 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.876490 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:58.892947 (+ 16457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:58.892955 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:58.892959 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:58.893006 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:58.896554 (+ 3548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:58.897284 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:58.897290 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:58.897294 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:58.897474 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:58.897939 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:58.897945 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:58.897948 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:58.898349 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:58.898514 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:58.898957 (+ 443us) server_negotiation.cc:299] Negotiation successful
0809 15:07:58.899201 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12827,"thread_start_us":12718,"threads_started":1}
I20260809 15:07:58.899609 4320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:58.850800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.24.183.195:42593)
0809 15:07:58.860644 (+ 9844us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:58.860664 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:58.860793 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:58.864151 (+ 3358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:58.864155 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:58.865870 (+ 1715us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:58.866221 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:58.866233 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.874498 (+ 8265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.874509 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:58.875789 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:58.875807 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.876554 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.876560 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:58.876727 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:58.877291 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:58.877314 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:58.890662 (+ 13348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:58.896709 (+ 6047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:58.896716 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:58.896720 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:58.897136 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:58.897609 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:58.897616 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:58.897620 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:58.897794 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:58.898463 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:58.898468 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:58.898634 (+ 166us) client_negotiation.cc:769] Sending connection context
0809 15:07:58.898782 (+ 148us) client_negotiation.cc:240] Negotiation successful
0809 15:07:58.899134 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9716,"mutex_wait_us":3,"thread_start_us":11972,"threads_started":1}
I20260809 15:07:58.900480 4305 heartbeater.cc:347] Connected to a master server at 127.24.183.254:45957
I20260809 15:07:58.900604 4305 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:58.901180 4305 heartbeater.cc:511] Master 127.24.183.254:45957 requested a full tablet report, sending...
I20260809 15:07:58.902275 2223 ts_manager.cc:194] Re-registered known tserver with Master: 2ed67a7ab72e4760917f53986f1a774a (127.24.183.195:40011)
I20260809 15:07:58.902468 4308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:58.850081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:39741 (local address 127.24.183.195:35471)
0809 15:07:58.857175 (+ 7094us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:58.857230 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:58.859884 (+ 2654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:58.864002 (+ 4118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:58.864010 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:58.865040 (+ 1030us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:07:58.866055 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:58.866086 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.867839 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.867848 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:58.871332 (+ 3484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:58.871347 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.879722 (+ 8375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.879729 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:58.879897 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:58.880443 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:07:58.880464 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:07:58.892017 (+ 11553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:07:58.898632 (+ 6615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:58.898638 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:58.898643 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:58.899156 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:58.900649 (+ 1493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:07:58.900654 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:07:58.900658 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:07:58.900864 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:07:58.901694 (+ 830us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:07:58.901698 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:07:58.901815 (+ 117us) client_negotiation.cc:769] Sending connection context
0809 15:07:58.901938 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 15:07:58.902183 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6500,"mutex_wait_us":53,"thread_start_us":766,"threads_started":1}
I20260809 15:07:58.902949 4309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:58.850793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:35471 (local address 127.24.183.252:39741)
0809 15:07:58.862505 (+ 11712us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:58.862512 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:58.862534 (+ 22us) server_negotiation.cc:407] Connection header received
0809 15:07:58.862599 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:58.862603 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:58.863301 (+ 698us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:58.863430 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:58.866329 (+ 2899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.867431 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.878834 (+ 11403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.879527 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.879655 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:58.892226 (+ 12571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:58.892233 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:58.892237 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:58.892283 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:58.898420 (+ 6137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:58.900286 (+ 1866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:58.900293 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:58.900297 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:58.900486 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:58.901003 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:58.901009 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:58.901012 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:58.901471 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:58.901625 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:58.902133 (+ 508us) server_negotiation.cc:299] Negotiation successful
0809 15:07:58.902383 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11595,"thread_start_us":11489,"threads_started":1}
I20260809 15:07:58.903333 2223 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.183.195@KRBTEST.COM'} at 127.24.183.195:42593
I20260809 15:07:58.903985 4303 heartbeater.cc:347] Connected to a master server at 127.24.183.252:39741
I20260809 15:07:58.904124 4303 heartbeater.cc:464] Registering TS with master...
I20260809 15:07:58.904587 4303 heartbeater.cc:511] Master 127.24.183.252:39741 requested a full tablet report, sending...
I20260809 15:07:58.906133 3514 ts_manager.cc:194] Registered new tserver with Master: 2ed67a7ab72e4760917f53986f1a774a (127.24.183.195:40011)
I20260809 15:07:58.919040 4340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:58.881061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39796 (local address 127.24.183.252:39741)
0809 15:07:58.896298 (+ 15237us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:58.896305 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:58.902641 (+ 6336us) server_negotiation.cc:407] Connection header received
0809 15:07:58.902950 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:58.902955 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:58.903032 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:58.903129 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:58.906024 (+ 2895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.907242 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.909068 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.909515 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.909625 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:58.913225 (+ 3600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:58.913232 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:58.913236 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:58.913276 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:58.916399 (+ 3123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:58.917041 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:58.917046 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:58.917050 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:58.917199 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:58.917633 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:58.917636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:58.917639 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:58.917888 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:58.918126 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:58.918443 (+ 317us) server_negotiation.cc:299] Negotiation successful
0809 15:07:58.918669 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15070,"thread_start_us":15248,"threads_started":1}
I20260809 15:07:58.920632 4337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:58.869429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46846 (local address 127.24.183.253:39987)
0809 15:07:58.878931 (+ 9502us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:58.878936 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:58.892073 (+ 13137us) server_negotiation.cc:407] Connection header received
0809 15:07:58.892137 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:58.892142 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:58.892226 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:58.892336 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:58.907469 (+ 15133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.908346 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.910044 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.910615 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.910753 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:58.913679 (+ 2926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:58.913684 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:58.913688 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:58.913728 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:58.916856 (+ 3128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:58.917669 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:58.917675 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:58.917678 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:58.917905 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:58.919090 (+ 1185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:58.919096 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:58.919099 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:58.919393 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:58.919636 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:58.920019 (+ 383us) server_negotiation.cc:299] Negotiation successful
0809 15:07:58.920254 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9353,"thread_start_us":9245,"threads_started":1}
I20260809 15:07:58.922041 4335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:58.867839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37300 (local address 127.24.183.254:45957)
0809 15:07:58.885838 (+ 17999us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:58.885843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:58.885860 (+ 17us) server_negotiation.cc:407] Connection header received
0809 15:07:58.885922 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:58.885926 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:58.886002 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:58.886103 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:58.907717 (+ 21614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.908877 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.910666 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.911521 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.911654 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:58.914832 (+ 3178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:58.914838 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:58.914843 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:58.914882 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:58.918432 (+ 3550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:58.919147 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:58.919153 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:58.919157 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:58.919321 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:58.920189 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:58.920193 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:58.920195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:58.920492 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:58.920704 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:58.921086 (+ 382us) server_negotiation.cc:299] Negotiation successful
0809 15:07:58.921608 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17886,"thread_start_us":18418,"threads_started":1}
I20260809 15:07:58.925848 2223 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37300:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:07:58.933528 2223 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:07:58.968000 4368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:58.958109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:46409 (local address 127.0.0.1:48562)
0809 15:07:58.959286 (+ 1177us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:58.959316 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:58.959424 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:58.961113 (+ 1689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:58.961120 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:58.961150 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:07:58.961548 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:58.961559 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.963336 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.963342 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:58.965687 (+ 2345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:58.965699 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.966994 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.967001 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:58.967286 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:58.967314 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 15:07:58.967398 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 15:07:58.967534 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1025,"mutex_wait_us":51,"thread_start_us":1742,"threads_started":1}
I20260809 15:07:58.969323 4366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:58.957492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:44531 (local address 127.0.0.1:37846)
0809 15:07:58.958648 (+ 1156us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:58.958669 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:58.958797 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:58.962386 (+ 3589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:58.962391 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:58.962416 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:07:58.962784 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:58.962793 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.964452 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.964458 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:58.966575 (+ 2117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:58.966590 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.968570 (+ 1980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.968577 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:58.968743 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:58.968777 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 15:07:58.968871 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 15:07:58.968994 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1018,"thread_start_us":300,"threads_started":1}
I20260809 15:07:58.969416 4367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:58.957744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37846 (local address 127.24.183.193:44531)
0809 15:07:58.959911 (+ 2167us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:58.959919 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:58.959947 (+ 28us) server_negotiation.cc:407] Connection header received
0809 15:07:58.960051 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:58.960059 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:58.961961 (+ 1902us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:58.962408 (+ 447us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:07:58.963061 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.964305 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.966792 (+ 2487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.968224 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.968379 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:58.968535 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:58.968945 (+ 410us) server_negotiation.cc:299] Negotiation successful
0809 15:07:58.969106 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2013,"thread_start_us":1867,"threads_started":1}
I20260809 15:07:58.969875 4371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:58.958660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48562 (local address 127.24.183.194:46409)
0809 15:07:58.959837 (+ 1177us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:58.959846 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:58.960238 (+ 392us) server_negotiation.cc:407] Connection header received
0809 15:07:58.960354 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:58.960361 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:58.960642 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:58.960850 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:07:58.961754 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.963211 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.965836 (+ 2625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.966786 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.967004 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:58.967197 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:58.967461 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 15:07:58.967634 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":984,"thread_start_us":1630,"threads_started":1}
I20260809 15:07:58.970577 4372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:58.960172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:40011 (local address 127.0.0.1:36426)
0809 15:07:58.962070 (+ 1898us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:58.962087 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:58.962206 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:58.962765 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:58.962773 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:58.962796 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:07:58.963197 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:58.963208 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.965556 (+ 2348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.965561 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:58.967750 (+ 2189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:58.967764 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.969796 (+ 2032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.969804 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:58.969973 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:58.970002 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 15:07:58.970083 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 15:07:58.970193 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1783,"mutex_wait_us":38,"thread_start_us":1644,"threads_started":1}
I20260809 15:07:58.972373 4373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:58.960431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36426 (local address 127.24.183.195:40011)
0809 15:07:58.961637 (+ 1206us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:58.961646 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:58.962195 (+ 549us) server_negotiation.cc:407] Connection header received
0809 15:07:58.962349 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:58.962356 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:58.962566 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:58.962752 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:07:58.963804 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.965413 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.968065 (+ 2652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:58.969520 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:58.969676 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:58.969830 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:58.971881 (+ 2051us) server_negotiation.cc:299] Negotiation successful
0809 15:07:58.972051 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1027,"thread_start_us":1111,"threads_started":1}
I20260809 15:07:58.973047 3722 tablet_service.cc:1467] Processing CreateTablet for tablet ac4b21dbb8ee444abe4fb7df0cb6f23f (DEFAULT_TABLE table=test-table [id=cfa1d9fa80074808954ac28ceda80064]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:07:58.973045 3977 tablet_service.cc:1467] Processing CreateTablet for tablet ac4b21dbb8ee444abe4fb7df0cb6f23f (DEFAULT_TABLE table=test-table [id=cfa1d9fa80074808954ac28ceda80064]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:07:58.974452 3977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac4b21dbb8ee444abe4fb7df0cb6f23f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:58.974474 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac4b21dbb8ee444abe4fb7df0cb6f23f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:58.977504 4203 tablet_service.cc:1467] Processing CreateTablet for tablet ac4b21dbb8ee444abe4fb7df0cb6f23f (DEFAULT_TABLE table=test-table [id=cfa1d9fa80074808954ac28ceda80064]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:07:58.978511 4203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac4b21dbb8ee444abe4fb7df0cb6f23f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:07:58.988749 4377 tablet_bootstrap.cc:492] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 10025c6198174f61b5495c94ea3a510e: Bootstrap starting.
I20260809 15:07:58.990579 4378 tablet_bootstrap.cc:492] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 2ed67a7ab72e4760917f53986f1a774a: Bootstrap starting.
I20260809 15:07:58.992177 4379 tablet_bootstrap.cc:492] T ac4b21dbb8ee444abe4fb7df0cb6f23f P cfc40b3492174e7cb2dfd14d21e72582: Bootstrap starting.
I20260809 15:07:58.992538 4377 tablet_bootstrap.cc:654] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 10025c6198174f61b5495c94ea3a510e: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:58.993845 4377 log.cc:824] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 10025c6198174f61b5495c94ea3a510e: Log is configured to *not* fsync() on all Append() calls
I20260809 15:07:58.993919 4378 tablet_bootstrap.cc:654] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 2ed67a7ab72e4760917f53986f1a774a: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:58.995067 4378 log.cc:824] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 2ed67a7ab72e4760917f53986f1a774a: Log is configured to *not* fsync() on all Append() calls
I20260809 15:07:58.995806 4379 tablet_bootstrap.cc:654] T ac4b21dbb8ee444abe4fb7df0cb6f23f P cfc40b3492174e7cb2dfd14d21e72582: Neither blocks nor log segments found. Creating new log.
I20260809 15:07:58.996994 4379 log.cc:824] T ac4b21dbb8ee444abe4fb7df0cb6f23f P cfc40b3492174e7cb2dfd14d21e72582: Log is configured to *not* fsync() on all Append() calls
I20260809 15:07:58.997131 4377 tablet_bootstrap.cc:492] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 10025c6198174f61b5495c94ea3a510e: No bootstrap required, opened a new log
I20260809 15:07:58.997424 4377 ts_tablet_manager.cc:1397] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 10025c6198174f61b5495c94ea3a510e: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 15:07:58.998565 4378 tablet_bootstrap.cc:492] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 2ed67a7ab72e4760917f53986f1a774a: No bootstrap required, opened a new log
I20260809 15:07:58.998832 4378 ts_tablet_manager.cc:1397] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 2ed67a7ab72e4760917f53986f1a774a: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260809 15:07:58.999362 4379 tablet_bootstrap.cc:492] T ac4b21dbb8ee444abe4fb7df0cb6f23f P cfc40b3492174e7cb2dfd14d21e72582: No bootstrap required, opened a new log
I20260809 15:07:58.999581 4379 ts_tablet_manager.cc:1397] T ac4b21dbb8ee444abe4fb7df0cb6f23f P cfc40b3492174e7cb2dfd14d21e72582: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 15:07:59.006953 4379 raft_consensus.cc:348] T ac4b21dbb8ee444abe4fb7df0cb6f23f P cfc40b3492174e7cb2dfd14d21e72582 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfc40b3492174e7cb2dfd14d21e72582" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 44531 } } peers { permanent_uuid: "10025c6198174f61b5495c94ea3a510e" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46409 } } peers { permanent_uuid: "2ed67a7ab72e4760917f53986f1a774a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 40011 } }
I20260809 15:07:59.007282 4379 raft_consensus.cc:374] T ac4b21dbb8ee444abe4fb7df0cb6f23f P cfc40b3492174e7cb2dfd14d21e72582 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:59.007383 4379 raft_consensus.cc:729] T ac4b21dbb8ee444abe4fb7df0cb6f23f P cfc40b3492174e7cb2dfd14d21e72582 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfc40b3492174e7cb2dfd14d21e72582, State: Initialized, Role: FOLLOWER
I20260809 15:07:59.007943 4379 consensus_queue.cc:260] T ac4b21dbb8ee444abe4fb7df0cb6f23f P cfc40b3492174e7cb2dfd14d21e72582 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfc40b3492174e7cb2dfd14d21e72582" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 44531 } } peers { permanent_uuid: "10025c6198174f61b5495c94ea3a510e" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46409 } } peers { permanent_uuid: "2ed67a7ab72e4760917f53986f1a774a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 40011 } }
I20260809 15:07:59.007817 4377 raft_consensus.cc:348] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 10025c6198174f61b5495c94ea3a510e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfc40b3492174e7cb2dfd14d21e72582" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 44531 } } peers { permanent_uuid: "10025c6198174f61b5495c94ea3a510e" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46409 } } peers { permanent_uuid: "2ed67a7ab72e4760917f53986f1a774a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 40011 } }
I20260809 15:07:59.008205 4377 raft_consensus.cc:374] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 10025c6198174f61b5495c94ea3a510e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:59.008306 4377 raft_consensus.cc:729] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 10025c6198174f61b5495c94ea3a510e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10025c6198174f61b5495c94ea3a510e, State: Initialized, Role: FOLLOWER
I20260809 15:07:59.008898 4377 consensus_queue.cc:260] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 10025c6198174f61b5495c94ea3a510e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfc40b3492174e7cb2dfd14d21e72582" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 44531 } } peers { permanent_uuid: "10025c6198174f61b5495c94ea3a510e" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46409 } } peers { permanent_uuid: "2ed67a7ab72e4760917f53986f1a774a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 40011 } }
I20260809 15:07:59.009094 4378 raft_consensus.cc:348] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 2ed67a7ab72e4760917f53986f1a774a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfc40b3492174e7cb2dfd14d21e72582" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 44531 } } peers { permanent_uuid: "10025c6198174f61b5495c94ea3a510e" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46409 } } peers { permanent_uuid: "2ed67a7ab72e4760917f53986f1a774a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 40011 } }
I20260809 15:07:59.009513 4378 raft_consensus.cc:374] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 2ed67a7ab72e4760917f53986f1a774a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:07:59.009609 4378 raft_consensus.cc:729] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 2ed67a7ab72e4760917f53986f1a774a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ed67a7ab72e4760917f53986f1a774a, State: Initialized, Role: FOLLOWER
I20260809 15:07:59.009670 4379 ts_tablet_manager.cc:1428] T ac4b21dbb8ee444abe4fb7df0cb6f23f P cfc40b3492174e7cb2dfd14d21e72582: Time spent starting tablet: real 0.010s user 0.008s sys 0.003s
I20260809 15:07:59.010325 4378 consensus_queue.cc:260] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 2ed67a7ab72e4760917f53986f1a774a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfc40b3492174e7cb2dfd14d21e72582" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 44531 } } peers { permanent_uuid: "10025c6198174f61b5495c94ea3a510e" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46409 } } peers { permanent_uuid: "2ed67a7ab72e4760917f53986f1a774a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 40011 } }
I20260809 15:07:59.015657 4377 ts_tablet_manager.cc:1428] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 10025c6198174f61b5495c94ea3a510e: Time spent starting tablet: real 0.018s user 0.018s sys 0.000s
I20260809 15:07:59.016335 4085 heartbeater.cc:503] Master 127.24.183.254:45957 was elected leader, sending a full tablet report...
I20260809 15:07:59.019470 4305 heartbeater.cc:503] Master 127.24.183.254:45957 was elected leader, sending a full tablet report...
I20260809 15:07:59.021250 4378 ts_tablet_manager.cc:1428] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 2ed67a7ab72e4760917f53986f1a774a: Time spent starting tablet: real 0.022s user 0.012s sys 0.008s
I20260809 15:07:59.086365 4384 raft_consensus.cc:482] T ac4b21dbb8ee444abe4fb7df0cb6f23f P cfc40b3492174e7cb2dfd14d21e72582 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:07:59.086664 4384 raft_consensus.cc:504] T ac4b21dbb8ee444abe4fb7df0cb6f23f P cfc40b3492174e7cb2dfd14d21e72582 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfc40b3492174e7cb2dfd14d21e72582" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 44531 } } peers { permanent_uuid: "10025c6198174f61b5495c94ea3a510e" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46409 } } peers { permanent_uuid: "2ed67a7ab72e4760917f53986f1a774a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 40011 } }
I20260809 15:07:59.088517 4384 leader_election.cc:288] T ac4b21dbb8ee444abe4fb7df0cb6f23f P cfc40b3492174e7cb2dfd14d21e72582 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10025c6198174f61b5495c94ea3a510e (127.24.183.194:46409), 2ed67a7ab72e4760917f53986f1a774a (127.24.183.195:40011)
I20260809 15:07:59.094998 4371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:59.088856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:33437 (local address 127.24.183.194:46409)
0809 15:07:59.089052 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:59.089058 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:59.089566 (+ 508us) server_negotiation.cc:407] Connection header received
0809 15:07:59.089861 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:59.089868 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:59.089960 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:59.090025 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:07:59.090679 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:59.091409 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:59.093283 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:59.094081 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:59.094235 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:59.094313 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:59.094604 (+ 291us) server_negotiation.cc:299] Negotiation successful
0809 15:07:59.094730 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 15:07:59.095099 4391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:59.088735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:46409 (local address 127.24.183.193:33437)
0809 15:07:59.089458 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:59.089483 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:59.089627 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:59.090206 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:59.090210 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:59.090231 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:07:59.090531 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:59.090538 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:59.091550 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:59.091554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:59.093102 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:59.093115 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:59.094292 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:59.094298 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:59.094440 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:59.094465 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 15:07:59.094556 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 15:07:59.094683 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"mutex_wait_us":22,"thread_start_us":350,"threads_started":1}
I20260809 15:07:59.095213 4373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:59.089639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:55817 (local address 127.24.183.195:40011)
0809 15:07:59.089909 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:59.089916 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:59.090178 (+ 262us) server_negotiation.cc:407] Connection header received
0809 15:07:59.090357 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:59.090363 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:59.090434 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:59.090508 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:07:59.091021 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:59.091889 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:59.093601 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:59.094355 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:59.094511 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:59.094587 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:59.094852 (+ 265us) server_negotiation.cc:299] Negotiation successful
0809 15:07:59.094979 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
I20260809 15:07:59.095507 4393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:59.089516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:40011 (local address 127.24.183.193:55817)
0809 15:07:59.090074 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0809 15:07:59.090088 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:07:59.090217 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:07:59.090550 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:07:59.090554 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:07:59.090578 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:07:59.090865 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:59.090874 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:59.092039 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:59.092044 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:59.093458 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:07:59.093470 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:07:59.094571 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:07:59.094578 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:07:59.094712 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:59.094735 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 15:07:59.094821 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 15:07:59.094951 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"mutex_wait_us":17,"thread_start_us":289,"threads_started":1}
I20260809 15:07:59.096213 4010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac4b21dbb8ee444abe4fb7df0cb6f23f" candidate_uuid: "cfc40b3492174e7cb2dfd14d21e72582" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10025c6198174f61b5495c94ea3a510e" is_pre_election: true
I20260809 15:07:59.096462 4236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac4b21dbb8ee444abe4fb7df0cb6f23f" candidate_uuid: "cfc40b3492174e7cb2dfd14d21e72582" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ed67a7ab72e4760917f53986f1a774a" is_pre_election: true
I20260809 15:07:59.096719 4010 raft_consensus.cc:2454] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 10025c6198174f61b5495c94ea3a510e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cfc40b3492174e7cb2dfd14d21e72582 in term 0.
I20260809 15:07:59.096959 4236 raft_consensus.cc:2454] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 2ed67a7ab72e4760917f53986f1a774a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cfc40b3492174e7cb2dfd14d21e72582 in term 0.
I20260809 15:07:59.097357 3635 leader_election.cc:302] T ac4b21dbb8ee444abe4fb7df0cb6f23f P cfc40b3492174e7cb2dfd14d21e72582 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10025c6198174f61b5495c94ea3a510e, cfc40b3492174e7cb2dfd14d21e72582; no voters:
I20260809 15:07:59.097715 4384 raft_consensus.cc:2781] T ac4b21dbb8ee444abe4fb7df0cb6f23f P cfc40b3492174e7cb2dfd14d21e72582 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:07:59.097870 4384 raft_consensus.cc:482] T ac4b21dbb8ee444abe4fb7df0cb6f23f P cfc40b3492174e7cb2dfd14d21e72582 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:07:59.097998 4384 raft_consensus.cc:3037] T ac4b21dbb8ee444abe4fb7df0cb6f23f P cfc40b3492174e7cb2dfd14d21e72582 [term 0 FOLLOWER]: Advancing to term 1
W20260809 15:07:59.099752 4307 tablet.cc:2369] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 2ed67a7ab72e4760917f53986f1a774a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:07:59.100630 4384 raft_consensus.cc:504] T ac4b21dbb8ee444abe4fb7df0cb6f23f P cfc40b3492174e7cb2dfd14d21e72582 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfc40b3492174e7cb2dfd14d21e72582" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 44531 } } peers { permanent_uuid: "10025c6198174f61b5495c94ea3a510e" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46409 } } peers { permanent_uuid: "2ed67a7ab72e4760917f53986f1a774a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 40011 } }
I20260809 15:07:59.101224 4384 leader_election.cc:288] T ac4b21dbb8ee444abe4fb7df0cb6f23f P cfc40b3492174e7cb2dfd14d21e72582 [CANDIDATE]: Term 1 election: Requested vote from peers 10025c6198174f61b5495c94ea3a510e (127.24.183.194:46409), 2ed67a7ab72e4760917f53986f1a774a (127.24.183.195:40011)
I20260809 15:07:59.101655 4010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac4b21dbb8ee444abe4fb7df0cb6f23f" candidate_uuid: "cfc40b3492174e7cb2dfd14d21e72582" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10025c6198174f61b5495c94ea3a510e"
I20260809 15:07:59.101667 4236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac4b21dbb8ee444abe4fb7df0cb6f23f" candidate_uuid: "cfc40b3492174e7cb2dfd14d21e72582" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ed67a7ab72e4760917f53986f1a774a"
I20260809 15:07:59.101852 4236 raft_consensus.cc:3037] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 2ed67a7ab72e4760917f53986f1a774a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:59.101831 4010 raft_consensus.cc:3037] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 10025c6198174f61b5495c94ea3a510e [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:07:59.104092 4010 raft_consensus.cc:2454] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 10025c6198174f61b5495c94ea3a510e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cfc40b3492174e7cb2dfd14d21e72582 in term 1.
I20260809 15:07:59.104187 4236 raft_consensus.cc:2454] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 2ed67a7ab72e4760917f53986f1a774a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cfc40b3492174e7cb2dfd14d21e72582 in term 1.
I20260809 15:07:59.104450 3635 leader_election.cc:302] T ac4b21dbb8ee444abe4fb7df0cb6f23f P cfc40b3492174e7cb2dfd14d21e72582 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10025c6198174f61b5495c94ea3a510e, cfc40b3492174e7cb2dfd14d21e72582; no voters:
I20260809 15:07:59.104737 4384 raft_consensus.cc:2781] T ac4b21dbb8ee444abe4fb7df0cb6f23f P cfc40b3492174e7cb2dfd14d21e72582 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:07:59.105165 4384 raft_consensus.cc:686] T ac4b21dbb8ee444abe4fb7df0cb6f23f P cfc40b3492174e7cb2dfd14d21e72582 [term 1 LEADER]: Becoming Leader. State: Replica: cfc40b3492174e7cb2dfd14d21e72582, State: Running, Role: LEADER
I20260809 15:07:59.105680 4384 consensus_queue.cc:237] T ac4b21dbb8ee444abe4fb7df0cb6f23f P cfc40b3492174e7cb2dfd14d21e72582 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfc40b3492174e7cb2dfd14d21e72582" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 44531 } } peers { permanent_uuid: "10025c6198174f61b5495c94ea3a510e" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46409 } } peers { permanent_uuid: "2ed67a7ab72e4760917f53986f1a774a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 40011 } }
I20260809 15:07:59.110908 2223 catalog_manager.cc:5495] T ac4b21dbb8ee444abe4fb7df0cb6f23f P cfc40b3492174e7cb2dfd14d21e72582 reported cstate change: term changed from 0 to 1, leader changed from <none> to cfc40b3492174e7cb2dfd14d21e72582 (127.24.183.193). New cstate: current_term: 1 leader_uuid: "cfc40b3492174e7cb2dfd14d21e72582" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfc40b3492174e7cb2dfd14d21e72582" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 44531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10025c6198174f61b5495c94ea3a510e" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ed67a7ab72e4760917f53986f1a774a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 40011 } health_report { overall_health: UNKNOWN } } }
W20260809 15:07:59.122071 3836 tablet.cc:2369] T ac4b21dbb8ee444abe4fb7df0cb6f23f P cfc40b3492174e7cb2dfd14d21e72582: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:07:59.159905 4367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:59.154975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37856 (local address 127.24.183.193:44531)
0809 15:07:59.155175 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:59.155181 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:59.155201 (+ 20us) server_negotiation.cc:407] Connection header received
0809 15:07:59.155291 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:59.155295 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:59.155378 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:59.155505 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:07:59.155947 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:59.156735 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:59.158031 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:59.158454 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:59.158546 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:59.158938 (+ 392us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:07:59.159431 (+ 493us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:07:59.159538 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:59.159577 (+ 39us) server_negotiation.cc:299] Negotiation successful
0809 15:07:59.159642 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 15:07:59.168023 4236 raft_consensus.cc:1261] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 2ed67a7ab72e4760917f53986f1a774a [term 1 FOLLOWER]: Refusing update from remote peer cfc40b3492174e7cb2dfd14d21e72582: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:07:59.168054 4010 raft_consensus.cc:1261] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 10025c6198174f61b5495c94ea3a510e [term 1 FOLLOWER]: Refusing update from remote peer cfc40b3492174e7cb2dfd14d21e72582: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:07:59.168932 4394 consensus_queue.cc:1035] T ac4b21dbb8ee444abe4fb7df0cb6f23f P cfc40b3492174e7cb2dfd14d21e72582 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ed67a7ab72e4760917f53986f1a774a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 40011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:07:59.169364 4384 consensus_queue.cc:1035] T ac4b21dbb8ee444abe4fb7df0cb6f23f P cfc40b3492174e7cb2dfd14d21e72582 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10025c6198174f61b5495c94ea3a510e" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:07:59.203459 2223 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:07:59.203730 2223 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:07:59.210064 2223 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2be4818e2d7f4e51ba9ff391f3aecb81: Sending DeleteTablet for 3 replicas of tablet ac4b21dbb8ee444abe4fb7df0cb6f23f
I20260809 15:07:59.211472 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet ac4b21dbb8ee444abe4fb7df0cb6f23f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:07:59 UTC) from {username='oryx', principal='oryx/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:37846
I20260809 15:07:59.211790 3977 tablet_service.cc:1514] Processing DeleteTablet for tablet ac4b21dbb8ee444abe4fb7df0cb6f23f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:07:59 UTC) from {username='oryx', principal='oryx/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:48562
I20260809 15:07:59.211804 4203 tablet_service.cc:1514] Processing DeleteTablet for tablet ac4b21dbb8ee444abe4fb7df0cb6f23f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:07:59 UTC) from {username='oryx', principal='oryx/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:36426
I20260809 15:07:59.236485 4416 tablet_replica.cc:330] T ac4b21dbb8ee444abe4fb7df0cb6f23f P cfc40b3492174e7cb2dfd14d21e72582: stopping tablet replica
I20260809 15:07:59.236943 4416 raft_consensus.cc:2229] T ac4b21dbb8ee444abe4fb7df0cb6f23f P cfc40b3492174e7cb2dfd14d21e72582 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:07:59.237613 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 2175
I20260809 15:07:59.237735 4416 raft_consensus.cc:2258] T ac4b21dbb8ee444abe4fb7df0cb6f23f P cfc40b3492174e7cb2dfd14d21e72582 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:07:59.238544 4418 tablet_replica.cc:330] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 2ed67a7ab72e4760917f53986f1a774a: stopping tablet replica
I20260809 15:07:59.238950 2269 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:07:59.238963 4418 raft_consensus.cc:2229] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 2ed67a7ab72e4760917f53986f1a774a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:07:59.239503 4418 raft_consensus.cc:2258] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 2ed67a7ab72e4760917f53986f1a774a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:07:59.243022 4416 ts_tablet_manager.cc:1905] T ac4b21dbb8ee444abe4fb7df0cb6f23f P cfc40b3492174e7cb2dfd14d21e72582: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:07:59.250746 4416 ts_tablet_manager.cc:1918] T ac4b21dbb8ee444abe4fb7df0cb6f23f P cfc40b3492174e7cb2dfd14d21e72582: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:07:59.250936 4416 log.cc:1196] T ac4b21dbb8ee444abe4fb7df0cb6f23f P cfc40b3492174e7cb2dfd14d21e72582: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-0/wal/wals/ac4b21dbb8ee444abe4fb7df0cb6f23f
I20260809 15:07:59.251549 4416 ts_tablet_manager.cc:1939] T ac4b21dbb8ee444abe4fb7df0cb6f23f P cfc40b3492174e7cb2dfd14d21e72582: Deleting consensus metadata
I20260809 15:07:59.252871 4418 ts_tablet_manager.cc:1905] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 2ed67a7ab72e4760917f53986f1a774a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:07:59.256228 4417 tablet_replica.cc:330] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 10025c6198174f61b5495c94ea3a510e: stopping tablet replica
I20260809 15:07:59.256732 4417 raft_consensus.cc:2229] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 10025c6198174f61b5495c94ea3a510e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:07:59.257316 4417 raft_consensus.cc:2258] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 10025c6198174f61b5495c94ea3a510e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:07:59.262122 4418 ts_tablet_manager.cc:1918] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 2ed67a7ab72e4760917f53986f1a774a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:07:59.262282 4418 log.cc:1196] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 2ed67a7ab72e4760917f53986f1a774a: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-2/wal/wals/ac4b21dbb8ee444abe4fb7df0cb6f23f
I20260809 15:07:59.262785 4418 ts_tablet_manager.cc:1939] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 2ed67a7ab72e4760917f53986f1a774a: Deleting consensus metadata
I20260809 15:07:59.265857 4417 ts_tablet_manager.cc:1905] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 10025c6198174f61b5495c94ea3a510e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:07:59.267210 4337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:59.214791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46862 (local address 127.24.183.253:39987)
0809 15:07:59.214975 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:59.214981 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:59.236543 (+ 21562us) server_negotiation.cc:407] Connection header received
0809 15:07:59.239773 (+ 3230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:59.239780 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:59.239883 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:59.239975 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:59.247817 (+ 7842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:59.249044 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:59.251888 (+ 2844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:59.252653 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:59.252758 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:59.260825 (+ 8067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:59.260832 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:59.260837 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:59.260881 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:59.264279 (+ 3398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:59.265145 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:59.265152 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:59.265155 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:59.265316 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:59.265704 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:59.265709 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:59.265713 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:59.266010 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:59.266246 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:59.266629 (+ 383us) server_negotiation.cc:299] Negotiation successful
0809 15:07:59.266852 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 15:07:59.277806 4417 ts_tablet_manager.cc:1918] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 10025c6198174f61b5495c94ea3a510e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:07:59.277984 4417 log.cc:1196] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 10025c6198174f61b5495c94ea3a510e: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-1/wal/wals/ac4b21dbb8ee444abe4fb7df0cb6f23f
I20260809 15:07:59.278575 4417 ts_tablet_manager.cc:1939] T ac4b21dbb8ee444abe4fb7df0cb6f23f P 10025c6198174f61b5495c94ea3a510e: Deleting consensus metadata
I20260809 15:07:59.283947 4340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:59.235708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39812 (local address 127.24.183.252:39741)
0809 15:07:59.235946 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:59.235954 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:59.267290 (+ 31336us) server_negotiation.cc:407] Connection header received
0809 15:07:59.267762 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:59.267769 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:59.267878 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:59.267968 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:59.269988 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:59.270829 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:59.272865 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:59.273583 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:59.273683 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:59.277203 (+ 3520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:59.277210 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:59.277214 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:59.277257 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:59.280842 (+ 3585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:59.281620 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:59.281626 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:59.281630 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:59.281806 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:59.282254 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:59.282260 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:59.282263 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:59.282653 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:59.282916 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:59.283250 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 15:07:59.283529 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 15:07:59.397562 2202 catalog_manager.cc:4841] TS 10025c6198174f61b5495c94ea3a510e (127.24.183.194:46409): tablet ac4b21dbb8ee444abe4fb7df0cb6f23f (table test-table [id=cfa1d9fa80074808954ac28ceda80064]) successfully deleted
I20260809 15:07:59.397989 2202 catalog_manager.cc:4841] TS 2ed67a7ab72e4760917f53986f1a774a (127.24.183.195:40011): tablet ac4b21dbb8ee444abe4fb7df0cb6f23f (table test-table [id=cfa1d9fa80074808954ac28ceda80064]) successfully deleted
I20260809 15:07:59.398186 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 2175
I20260809 15:07:59.398245 2202 catalog_manager.cc:4841] TS cfc40b3492174e7cb2dfd14d21e72582 (127.24.183.193:44531): tablet ac4b21dbb8ee444abe4fb7df0cb6f23f (table test-table [id=cfa1d9fa80074808954ac28ceda80064]) successfully deleted
I20260809 15:07:59.455327 4337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:59.427064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46878 (local address 127.24.183.253:39987)
0809 15:07:59.427249 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:59.427256 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:59.437349 (+ 10093us) server_negotiation.cc:407] Connection header received
0809 15:07:59.437555 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:59.437561 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:59.437841 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:59.437939 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:59.440214 (+ 2275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:59.441443 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:59.443389 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:59.444135 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:59.444222 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:59.447960 (+ 3738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:59.447966 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:59.447971 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:59.448009 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:59.451973 (+ 3964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:59.452896 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:59.452901 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:59.452904 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:59.453075 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:59.453503 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:59.453508 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:59.453512 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:59.453951 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:59.454181 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:59.454776 (+ 595us) server_negotiation.cc:299] Negotiation successful
0809 15:07:59.454997 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 15:07:59.455572 4340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:07:59.426639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39824 (local address 127.24.183.252:39741)
0809 15:07:59.429393 (+ 2754us) server_negotiation.cc:206] Beginning negotiation
0809 15:07:59.429401 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:07:59.438027 (+ 8626us) server_negotiation.cc:407] Connection header received
0809 15:07:59.438272 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:07:59.438289 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:07:59.438380 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:07:59.438452 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:07:59.441351 (+ 2899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:59.442397 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:59.444388 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:07:59.445322 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:07:59.445431 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:07:59.448492 (+ 3061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:07:59.448498 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:07:59.448503 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:07:59.448552 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:07:59.451836 (+ 3284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:59.453199 (+ 1363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:59.453204 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:59.453207 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:59.453370 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:07:59.453815 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:07:59.453820 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:07:59.453824 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:07:59.454195 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:07:59.454423 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0809 15:07:59.455019 (+ 596us) server_negotiation.cc:299] Negotiation successful
0809 15:07:59.455241 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2561}
I20260809 15:08:00.188422 4436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:00.187483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.24.183.195:52241)
0809 15:08:00.188031 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:00.188179 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":297,"threads_started":1}
W20260809 15:08:00.189049 4305 heartbeater.cc:644] Failed to heartbeat to 127.24.183.254:45957 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.183.254:45957: connect: Connection refused (error 111)
I20260809 15:08:00.196486 4440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:00.192259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.24.183.194:47035)
0809 15:08:00.196183 (+ 3924us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:00.196309 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3796,"thread_start_us":3680,"threads_started":1}
W20260809 15:08:00.196929 4085 heartbeater.cc:644] Failed to heartbeat to 127.24.183.254:45957 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.183.254:45957: connect: Connection refused (error 111)
I20260809 15:08:00.198045 4441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:00.197297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.24.183.193:52451)
0809 15:08:00.197748 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:00.197889 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":240,"threads_started":1}
W20260809 15:08:00.198405 3828 heartbeater.cc:644] Failed to heartbeat to 127.24.183.254:45957 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.183.254:45957: connect: Connection refused (error 111)
I20260809 15:08:00.710961 4446 raft_consensus.cc:482] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2be4818e2d7f4e51ba9ff391f3aecb81)
I20260809 15:08:00.710955 4445 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2be4818e2d7f4e51ba9ff391f3aecb81)
I20260809 15:08:00.711231 4446 raft_consensus.cc:504] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: false } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: false } }
I20260809 15:08:00.711234 4445 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: false } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: false } }
I20260809 15:08:00.712582 4446 leader_election.cc:288] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2be4818e2d7f4e51ba9ff391f3aecb81 (127.24.183.254:45957), eb2109a9820942409f99c6897b415642 (127.24.183.252:39741)
I20260809 15:08:00.712606 4445 leader_election.cc:288] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2be4818e2d7f4e51ba9ff391f3aecb81 (127.24.183.254:45957), d835c269415b4f52b71dcbac3af1a081 (127.24.183.253:39987)
I20260809 15:08:00.726261 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:00.712733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.0.0.1:37762)
0809 15:08:00.725769 (+ 13036us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:00.725948 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12898,"threads_started":1}
W20260809 15:08:00.726788 3161 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.183.254:45957: connect: Connection refused (error 111)
I20260809 15:08:00.730379 4447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:00.712735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.0.0.1:37768)
0809 15:08:00.729906 (+ 17171us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:00.730138 (+ 232us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17029,"threads_started":1}
W20260809 15:08:00.730962 3488 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.183.254:45957: connect: Connection refused (error 111)
I20260809 15:08:00.733783 4452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:00.733031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.0.0.1:37784)
0809 15:08:00.733252 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:00.733532 (+ 280us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260809 15:08:00.733935 3488 leader_election.cc:334] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2be4818e2d7f4e51ba9ff391f3aecb81 (127.24.183.254:45957): Network error: Client connection negotiation failed: client connection to 127.24.183.254:45957: connect: Connection refused (error 111)
I20260809 15:08:00.735569 4451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:00.732296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.0.0.1:37774)
0809 15:08:00.734931 (+ 2635us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:00.735229 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2511}
W20260809 15:08:00.735783 3161 leader_election.cc:334] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2be4818e2d7f4e51ba9ff391f3aecb81 (127.24.183.254:45957): Network error: Client connection negotiation failed: client connection to 127.24.183.254:45957: connect: Connection refused (error 111)
I20260809 15:08:00.738937 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:00.713026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:39741 (local address 127.0.0.1:39830)
0809 15:08:00.727623 (+ 14597us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:00.727643 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:00.728111 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:00.731171 (+ 3060us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:00.731176 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:00.731203 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:08:00.731513 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:00.731526 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:00.734953 (+ 3427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:00.734958 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:00.736802 (+ 1844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:00.736817 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:00.738110 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:00.738116 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:00.738313 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:00.738349 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 15:08:00.738447 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 15:08:00.738585 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14495,"mutex_wait_us":24,"thread_start_us":21639,"threads_started":1}
I20260809 15:08:00.738973 4456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:00.714845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39830 (local address 127.24.183.252:39741)
0809 15:08:00.730819 (+ 15974us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:00.730826 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:00.730848 (+ 22us) server_negotiation.cc:407] Connection header received
0809 15:08:00.730926 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:00.730932 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:00.731019 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:00.731127 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:08:00.733309 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:00.734732 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:00.736954 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:00.737941 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:00.738105 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:00.738171 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:00.738491 (+ 320us) server_negotiation.cc:299] Negotiation successful
0809 15:08:00.738601 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":15839,"thread_start_us":16425,"threads_started":1}
I20260809 15:08:00.739761 3539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d835c269415b4f52b71dcbac3af1a081" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "eb2109a9820942409f99c6897b415642" is_pre_election: true
I20260809 15:08:00.740191 3539 raft_consensus.cc:2454] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d835c269415b4f52b71dcbac3af1a081 in term 1.
I20260809 15:08:00.740862 4447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:00.713204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:39987 (local address 127.0.0.1:46884)
0809 15:08:00.732327 (+ 19123us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:00.732345 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:00.732474 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:00.735034 (+ 2560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:00.735039 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:00.735067 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:08:00.735363 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:00.735375 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:00.736820 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:00.736825 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:00.738596 (+ 1771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:00.738611 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:00.740217 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:00.740223 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:00.740366 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:00.740392 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 15:08:00.740467 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 15:08:00.740605 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17284,"mutex_wait_us":48,"thread_start_us":19530,"threads_started":1}
I20260809 15:08:00.740765 3163 leader_election.cc:302] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d835c269415b4f52b71dcbac3af1a081, eb2109a9820942409f99c6897b415642; no voters: 2be4818e2d7f4e51ba9ff391f3aecb81
I20260809 15:08:00.741189 4446 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 15:08:00.741312 4446 raft_consensus.cc:482] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2be4818e2d7f4e51ba9ff391f3aecb81)
I20260809 15:08:00.741449 4446 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:08:00.743752 4455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:00.713579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46884 (local address 127.24.183.253:39987)
0809 15:08:00.734739 (+ 21160us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:00.734749 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:00.734770 (+ 21us) server_negotiation.cc:407] Connection header received
0809 15:08:00.734833 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:00.734837 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:00.734911 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:00.735025 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:08:00.735626 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:00.736692 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:00.738767 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:00.740043 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:00.740209 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:00.740279 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:00.740533 (+ 254us) server_negotiation.cc:299] Negotiation successful
0809 15:08:00.740661 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":21023,"thread_start_us":20811,"threads_started":1}
I20260809 15:08:00.743773 4446 raft_consensus.cc:504] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: false } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: false } }
I20260809 15:08:00.744467 4446 leader_election.cc:288] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [CANDIDATE]: Term 2 election: Requested vote from peers 2be4818e2d7f4e51ba9ff391f3aecb81 (127.24.183.254:45957), eb2109a9820942409f99c6897b415642 (127.24.183.252:39741)
I20260809 15:08:00.744982 3539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d835c269415b4f52b71dcbac3af1a081" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "eb2109a9820942409f99c6897b415642"
I20260809 15:08:00.745146 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:00.744671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.0.0.1:37792)
0809 15:08:00.744837 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:00.744943 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 15:08:00.745105 3199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eb2109a9820942409f99c6897b415642" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d835c269415b4f52b71dcbac3af1a081" is_pre_election: true
I20260809 15:08:00.745275 3539 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:08:00.745500 3199 raft_consensus.cc:2379] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb2109a9820942409f99c6897b415642 in current term 2: Already voted for candidate d835c269415b4f52b71dcbac3af1a081 in this term.
I20260809 15:08:00.746142 3487 leader_election.cc:302] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb2109a9820942409f99c6897b415642; no voters: 2be4818e2d7f4e51ba9ff391f3aecb81, d835c269415b4f52b71dcbac3af1a081
I20260809 15:08:00.748147 3539 raft_consensus.cc:2454] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d835c269415b4f52b71dcbac3af1a081 in term 2.
I20260809 15:08:00.748340 4445 raft_consensus.cc:2726] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:08:00.748633 3163 leader_election.cc:302] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d835c269415b4f52b71dcbac3af1a081, eb2109a9820942409f99c6897b415642; no voters:
I20260809 15:08:00.748947 4446 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:08:00.749442 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:00.749001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.0.0.1:37796)
0809 15:08:00.749156 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:00.749251 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 15:08:00.749497 4446 raft_consensus.cc:686] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [term 2 LEADER]: Becoming Leader. State: Replica: d835c269415b4f52b71dcbac3af1a081, State: Running, Role: LEADER
W20260809 15:08:00.749728 3161 leader_election.cc:334] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 2be4818e2d7f4e51ba9ff391f3aecb81 (127.24.183.254:45957): Network error: Client connection negotiation failed: client connection to 127.24.183.254:45957: connect: Connection refused (error 111)
I20260809 15:08:00.749997 4446 consensus_queue.cc:237] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: false } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: false } }
I20260809 15:08:00.752596 4462 sys_catalog.cc:455] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d835c269415b4f52b71dcbac3af1a081. Latest consensus state: current_term: 2 leader_uuid: "d835c269415b4f52b71dcbac3af1a081" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: false } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: false } } }
I20260809 15:08:00.753000 4462 sys_catalog.cc:458] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [sys.catalog]: This master's current role is: LEADER
I20260809 15:08:00.753727 4465 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:08:00.757956 4465 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:08:00.758852 4465 catalog_manager.cc:1259] Loaded cluster ID: 8879a6c828534eb6bb5186bd3f176cb4
I20260809 15:08:00.758934 4465 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:08:00.760187 4465 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:08:00.761029 4465 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081: Loaded TSK: 0
I20260809 15:08:00.761914 4465 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:08:00.791769 3185 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46878:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:08:00.799808 3185 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:08:00.804442 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:00.803957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.0.0.1:37814)
0809 15:08:00.804164 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:00.804275 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 15:08:00.804565 3539 raft_consensus.cc:1261] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [term 2 FOLLOWER]: Refusing update from remote peer d835c269415b4f52b71dcbac3af1a081: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260809 15:08:00.805459 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:00.805068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.0.0.1:37828)
0809 15:08:00.805201 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:00.805280 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 15:08:00.805347 4462 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260809 15:08:00.805676 3161 consensus_peers.cc:487] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 -> Peer 2be4818e2d7f4e51ba9ff391f3aecb81 (127.24.183.254:45957): Couldn't send request to peer 2be4818e2d7f4e51ba9ff391f3aecb81. Status: Network error: Client connection negotiation failed: client connection to 127.24.183.254:45957: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 15:08:00.807910 4445 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d835c269415b4f52b71dcbac3af1a081. Latest consensus state: current_term: 2 leader_uuid: "d835c269415b4f52b71dcbac3af1a081" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: false } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: false } } }
I20260809 15:08:00.808187 4445 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:08:00.809870 4462 sys_catalog.cc:455] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "d835c269415b4f52b71dcbac3af1a081" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: false } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: false } } }
I20260809 15:08:00.810168 4462 sys_catalog.cc:458] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081 [sys.catalog]: This master's current role is: LEADER
I20260809 15:08:00.810495 4445 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "d835c269415b4f52b71dcbac3af1a081" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2be4818e2d7f4e51ba9ff391f3aecb81" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45957 } } peers { permanent_uuid: "d835c269415b4f52b71dcbac3af1a081" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 39987 } attrs { promote: false } } peers { permanent_uuid: "eb2109a9820942409f99c6897b415642" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39741 } attrs { promote: false } } }
I20260809 15:08:00.810762 4445 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb2109a9820942409f99c6897b415642 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:08:00.811225 4467 mvcc.cc:205] Tried to move back new op lower bound from 7316635978968109056 to 7316635978756108288. Current Snapshot: MvccSnapshot[applied={T|T < 7316635978968109056 or (T in {7316635978968109056})}]
I20260809 15:08:00.838574 4475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:00.818246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36428 (local address 127.24.183.195:40011)
0809 15:08:00.832340 (+ 14094us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:00.832347 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:00.832369 (+ 22us) server_negotiation.cc:407] Connection header received
0809 15:08:00.832445 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:00.832450 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:00.832547 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:00.833110 (+ 563us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:08:00.833271 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:00.834364 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:00.836455 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:00.837587 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:00.837739 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:00.837818 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:00.838120 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 15:08:00.838276 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13964,"thread_start_us":14451,"threads_started":1}
I20260809 15:08:00.838608 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:00.818081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:40011 (local address 127.0.0.1:36428)
0809 15:08:00.818266 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:00.818286 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:00.818389 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:00.832799 (+ 14410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:00.832805 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:00.832834 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:08:00.832996 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:00.833003 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:00.834520 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:00.834524 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:00.836078 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:00.836090 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:00.837768 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:00.837774 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:00.837946 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:00.837980 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 15:08:00.838084 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 15:08:00.838227 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":24}
I20260809 15:08:00.839383 4478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:00.819123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:44531 (local address 127.0.0.1:37860)
0809 15:08:00.831771 (+ 12648us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:00.831790 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:00.831936 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:00.832997 (+ 1061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:00.833002 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:00.833031 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:08:00.833388 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:00.833399 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:00.834708 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:00.834712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:00.836745 (+ 2033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:00.836760 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:00.838560 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:00.838568 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:00.838757 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:00.838790 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 15:08:00.838880 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 15:08:00.839002 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12492,"mutex_wait_us":1,"thread_start_us":12250,"threads_started":1}
I20260809 15:08:00.839890 4203 tablet_service.cc:1467] Processing CreateTablet for tablet 655a85efa6c44eecb2bc8ab1aea4c8ba (DEFAULT_TABLE table=test-table [id=633be6ddb4c849a2a176ff260b2f62d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:08:00.840210 4481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:00.819288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37860 (local address 127.24.183.193:44531)
0809 15:08:00.832413 (+ 13125us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:00.832421 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:00.832443 (+ 22us) server_negotiation.cc:407] Connection header received
0809 15:08:00.832527 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:00.832532 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:00.832614 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:00.832715 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:08:00.833570 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:00.834581 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:00.836939 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:00.838123 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:00.838302 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:00.838375 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:00.839748 (+ 1373us) server_negotiation.cc:299] Negotiation successful
0809 15:08:00.839901 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12976,"thread_start_us":13528,"threads_started":1}
I20260809 15:08:00.840438 4203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 655a85efa6c44eecb2bc8ab1aea4c8ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:08:00.840956 3722 tablet_service.cc:1467] Processing CreateTablet for tablet 655a85efa6c44eecb2bc8ab1aea4c8ba (DEFAULT_TABLE table=test-table [id=633be6ddb4c849a2a176ff260b2f62d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:08:00.841476 3722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 655a85efa6c44eecb2bc8ab1aea4c8ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:08:00.841718 4451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:00.818429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:46409 (local address 127.0.0.1:48576)
0809 15:08:00.818747 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:00.818772 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:00.818879 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:00.834493 (+ 15614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:00.834497 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:00.834520 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:08:00.834963 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:00.834976 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:00.836974 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:00.836979 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:00.839037 (+ 2058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:00.839050 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:00.840975 (+ 1925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:00.840982 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:00.841180 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:00.841212 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 15:08:00.841282 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 15:08:00.841419 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220}
I20260809 15:08:00.842540 4477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:00.818759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48576 (local address 127.24.183.194:46409)
0809 15:08:00.834195 (+ 15436us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:00.834202 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:00.834218 (+ 16us) server_negotiation.cc:407] Connection header received
0809 15:08:00.834285 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:00.834289 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:00.834347 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:00.834439 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:08:00.835792 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:00.836819 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:00.839246 (+ 2427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:00.840726 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:00.841970 (+ 1244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:00.842051 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:00.842105 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 15:08:00.842246 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15307,"thread_start_us":15779,"threads_started":1}
I20260809 15:08:00.843252 3977 tablet_service.cc:1467] Processing CreateTablet for tablet 655a85efa6c44eecb2bc8ab1aea4c8ba (DEFAULT_TABLE table=test-table [id=633be6ddb4c849a2a176ff260b2f62d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:08:00.843708 3977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 655a85efa6c44eecb2bc8ab1aea4c8ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:08:00.852531 4484 tablet_bootstrap.cc:492] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 2ed67a7ab72e4760917f53986f1a774a: Bootstrap starting.
I20260809 15:08:00.854867 4484 tablet_bootstrap.cc:654] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 2ed67a7ab72e4760917f53986f1a774a: Neither blocks nor log segments found. Creating new log.
I20260809 15:08:00.855327 4490 tablet_bootstrap.cc:492] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 10025c6198174f61b5495c94ea3a510e: Bootstrap starting.
I20260809 15:08:00.857182 4490 tablet_bootstrap.cc:654] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 10025c6198174f61b5495c94ea3a510e: Neither blocks nor log segments found. Creating new log.
I20260809 15:08:00.858928 4489 tablet_bootstrap.cc:492] T 655a85efa6c44eecb2bc8ab1aea4c8ba P cfc40b3492174e7cb2dfd14d21e72582: Bootstrap starting.
I20260809 15:08:00.860407 4489 tablet_bootstrap.cc:654] T 655a85efa6c44eecb2bc8ab1aea4c8ba P cfc40b3492174e7cb2dfd14d21e72582: Neither blocks nor log segments found. Creating new log.
I20260809 15:08:00.860999 4490 tablet_bootstrap.cc:492] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 10025c6198174f61b5495c94ea3a510e: No bootstrap required, opened a new log
I20260809 15:08:00.861127 4490 ts_tablet_manager.cc:1397] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 10025c6198174f61b5495c94ea3a510e: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260809 15:08:00.861761 4490 raft_consensus.cc:348] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 10025c6198174f61b5495c94ea3a510e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed67a7ab72e4760917f53986f1a774a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 40011 } } peers { permanent_uuid: "10025c6198174f61b5495c94ea3a510e" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46409 } } peers { permanent_uuid: "cfc40b3492174e7cb2dfd14d21e72582" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 44531 } }
I20260809 15:08:00.861953 4490 raft_consensus.cc:374] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 10025c6198174f61b5495c94ea3a510e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:08:00.862035 4490 raft_consensus.cc:729] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 10025c6198174f61b5495c94ea3a510e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10025c6198174f61b5495c94ea3a510e, State: Initialized, Role: FOLLOWER
I20260809 15:08:00.862319 4489 tablet_bootstrap.cc:492] T 655a85efa6c44eecb2bc8ab1aea4c8ba P cfc40b3492174e7cb2dfd14d21e72582: No bootstrap required, opened a new log
I20260809 15:08:00.862449 4489 ts_tablet_manager.cc:1397] T 655a85efa6c44eecb2bc8ab1aea4c8ba P cfc40b3492174e7cb2dfd14d21e72582: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 15:08:00.862303 4490 consensus_queue.cc:260] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 10025c6198174f61b5495c94ea3a510e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed67a7ab72e4760917f53986f1a774a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 40011 } } peers { permanent_uuid: "10025c6198174f61b5495c94ea3a510e" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46409 } } peers { permanent_uuid: "cfc40b3492174e7cb2dfd14d21e72582" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 44531 } }
I20260809 15:08:00.863111 4489 raft_consensus.cc:348] T 655a85efa6c44eecb2bc8ab1aea4c8ba P cfc40b3492174e7cb2dfd14d21e72582 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed67a7ab72e4760917f53986f1a774a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 40011 } } peers { permanent_uuid: "10025c6198174f61b5495c94ea3a510e" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46409 } } peers { permanent_uuid: "cfc40b3492174e7cb2dfd14d21e72582" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 44531 } }
I20260809 15:08:00.863343 4484 tablet_bootstrap.cc:492] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 2ed67a7ab72e4760917f53986f1a774a: No bootstrap required, opened a new log
I20260809 15:08:00.863328 4489 raft_consensus.cc:374] T 655a85efa6c44eecb2bc8ab1aea4c8ba P cfc40b3492174e7cb2dfd14d21e72582 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:08:00.863463 4484 ts_tablet_manager.cc:1397] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 2ed67a7ab72e4760917f53986f1a774a: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260809 15:08:00.863435 4489 raft_consensus.cc:729] T 655a85efa6c44eecb2bc8ab1aea4c8ba P cfc40b3492174e7cb2dfd14d21e72582 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfc40b3492174e7cb2dfd14d21e72582, State: Initialized, Role: FOLLOWER
I20260809 15:08:00.863731 4489 consensus_queue.cc:260] T 655a85efa6c44eecb2bc8ab1aea4c8ba P cfc40b3492174e7cb2dfd14d21e72582 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed67a7ab72e4760917f53986f1a774a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 40011 } } peers { permanent_uuid: "10025c6198174f61b5495c94ea3a510e" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46409 } } peers { permanent_uuid: "cfc40b3492174e7cb2dfd14d21e72582" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 44531 } }
I20260809 15:08:00.864269 4484 raft_consensus.cc:348] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 2ed67a7ab72e4760917f53986f1a774a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed67a7ab72e4760917f53986f1a774a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 40011 } } peers { permanent_uuid: "10025c6198174f61b5495c94ea3a510e" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46409 } } peers { permanent_uuid: "cfc40b3492174e7cb2dfd14d21e72582" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 44531 } }
I20260809 15:08:00.864490 4484 raft_consensus.cc:374] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 2ed67a7ab72e4760917f53986f1a774a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:08:00.864585 4484 raft_consensus.cc:729] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 2ed67a7ab72e4760917f53986f1a774a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ed67a7ab72e4760917f53986f1a774a, State: Initialized, Role: FOLLOWER
I20260809 15:08:00.864832 4484 consensus_queue.cc:260] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 2ed67a7ab72e4760917f53986f1a774a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed67a7ab72e4760917f53986f1a774a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 40011 } } peers { permanent_uuid: "10025c6198174f61b5495c94ea3a510e" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46409 } } peers { permanent_uuid: "cfc40b3492174e7cb2dfd14d21e72582" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 44531 } }
I20260809 15:08:00.865527 4490 ts_tablet_manager.cc:1428] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 10025c6198174f61b5495c94ea3a510e: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260809 15:08:00.868894 4087 heartbeater.cc:503] Master 127.24.183.253:39987 was elected leader, sending a full tablet report...
I20260809 15:08:00.874393 4484 ts_tablet_manager.cc:1428] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 2ed67a7ab72e4760917f53986f1a774a: Time spent starting tablet: real 0.011s user 0.003s sys 0.007s
I20260809 15:08:00.874933 4499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:00.863971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.24.183.194:44435)
0809 15:08:00.874193 (+ 10222us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:00.874383 (+ 190us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10069,"thread_start_us":10564,"threads_started":1}
I20260809 15:08:00.876673 4499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:00.875651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.24.183.194:51599)
0809 15:08:00.876379 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:00.876486 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":625,"threads_started":1}
I20260809 15:08:00.880831 4306 heartbeater.cc:503] Master 127.24.183.253:39987 was elected leader, sending a full tablet report...
I20260809 15:08:00.882341 4504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:00.874927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.24.183.195:35815)
0809 15:08:00.881167 (+ 6240us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:00.881318 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6087,"threads_started":1}
I20260809 15:08:00.882448 4489 ts_tablet_manager.cc:1428] T 655a85efa6c44eecb2bc8ab1aea4c8ba P cfc40b3492174e7cb2dfd14d21e72582: Time spent starting tablet: real 0.020s user 0.009s sys 0.012s
I20260809 15:08:00.884047 4504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:00.882984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.24.183.195:41349)
0809 15:08:00.883821 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:00.883907 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":738}
I20260809 15:08:00.887943 3835 heartbeater.cc:503] Master 127.24.183.253:39987 was elected leader, sending a full tablet report...
I20260809 15:08:00.889070 4506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:00.882706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.24.183.193:35247)
0809 15:08:00.888625 (+ 5919us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:00.888799 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5797,"thread_start_us":5600,"threads_started":1}
I20260809 15:08:00.890036 4506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:00.889673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.24.183.193:51511)
0809 15:08:00.889863 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:00.889929 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
W20260809 15:08:00.978235 4089 tablet.cc:2369] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 10025c6198174f61b5495c94ea3a510e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:08:01.113637 4498 raft_consensus.cc:482] T 655a85efa6c44eecb2bc8ab1aea4c8ba P cfc40b3492174e7cb2dfd14d21e72582 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:08:01.113875 4498 raft_consensus.cc:504] T 655a85efa6c44eecb2bc8ab1aea4c8ba P cfc40b3492174e7cb2dfd14d21e72582 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed67a7ab72e4760917f53986f1a774a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 40011 } } peers { permanent_uuid: "10025c6198174f61b5495c94ea3a510e" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46409 } } peers { permanent_uuid: "cfc40b3492174e7cb2dfd14d21e72582" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 44531 } }
I20260809 15:08:01.114562 4498 leader_election.cc:288] T 655a85efa6c44eecb2bc8ab1aea4c8ba P cfc40b3492174e7cb2dfd14d21e72582 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ed67a7ab72e4760917f53986f1a774a (127.24.183.195:40011), 10025c6198174f61b5495c94ea3a510e (127.24.183.194:46409)
I20260809 15:08:01.115222 4236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "655a85efa6c44eecb2bc8ab1aea4c8ba" candidate_uuid: "cfc40b3492174e7cb2dfd14d21e72582" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ed67a7ab72e4760917f53986f1a774a" is_pre_election: true
I20260809 15:08:01.115306 4010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "655a85efa6c44eecb2bc8ab1aea4c8ba" candidate_uuid: "cfc40b3492174e7cb2dfd14d21e72582" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10025c6198174f61b5495c94ea3a510e" is_pre_election: true
I20260809 15:08:01.115475 4236 raft_consensus.cc:2454] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 2ed67a7ab72e4760917f53986f1a774a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cfc40b3492174e7cb2dfd14d21e72582 in term 0.
I20260809 15:08:01.115589 4010 raft_consensus.cc:2454] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 10025c6198174f61b5495c94ea3a510e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cfc40b3492174e7cb2dfd14d21e72582 in term 0.
I20260809 15:08:01.116021 3635 leader_election.cc:302] T 655a85efa6c44eecb2bc8ab1aea4c8ba P cfc40b3492174e7cb2dfd14d21e72582 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ed67a7ab72e4760917f53986f1a774a, cfc40b3492174e7cb2dfd14d21e72582; no voters:
I20260809 15:08:01.116279 4498 raft_consensus.cc:2781] T 655a85efa6c44eecb2bc8ab1aea4c8ba P cfc40b3492174e7cb2dfd14d21e72582 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:08:01.116384 4498 raft_consensus.cc:482] T 655a85efa6c44eecb2bc8ab1aea4c8ba P cfc40b3492174e7cb2dfd14d21e72582 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:08:01.116464 4498 raft_consensus.cc:3037] T 655a85efa6c44eecb2bc8ab1aea4c8ba P cfc40b3492174e7cb2dfd14d21e72582 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:08:01.118602 4498 raft_consensus.cc:504] T 655a85efa6c44eecb2bc8ab1aea4c8ba P cfc40b3492174e7cb2dfd14d21e72582 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed67a7ab72e4760917f53986f1a774a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 40011 } } peers { permanent_uuid: "10025c6198174f61b5495c94ea3a510e" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46409 } } peers { permanent_uuid: "cfc40b3492174e7cb2dfd14d21e72582" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 44531 } }
I20260809 15:08:01.119082 4498 leader_election.cc:288] T 655a85efa6c44eecb2bc8ab1aea4c8ba P cfc40b3492174e7cb2dfd14d21e72582 [CANDIDATE]: Term 1 election: Requested vote from peers 2ed67a7ab72e4760917f53986f1a774a (127.24.183.195:40011), 10025c6198174f61b5495c94ea3a510e (127.24.183.194:46409)
I20260809 15:08:01.119601 4236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "655a85efa6c44eecb2bc8ab1aea4c8ba" candidate_uuid: "cfc40b3492174e7cb2dfd14d21e72582" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ed67a7ab72e4760917f53986f1a774a"
I20260809 15:08:01.119726 4010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "655a85efa6c44eecb2bc8ab1aea4c8ba" candidate_uuid: "cfc40b3492174e7cb2dfd14d21e72582" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10025c6198174f61b5495c94ea3a510e"
I20260809 15:08:01.119822 4236 raft_consensus.cc:3037] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 2ed67a7ab72e4760917f53986f1a774a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:08:01.119933 4010 raft_consensus.cc:3037] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 10025c6198174f61b5495c94ea3a510e [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:08:01.121841 4236 raft_consensus.cc:2454] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 2ed67a7ab72e4760917f53986f1a774a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cfc40b3492174e7cb2dfd14d21e72582 in term 1.
I20260809 15:08:01.121973 4010 raft_consensus.cc:2454] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 10025c6198174f61b5495c94ea3a510e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cfc40b3492174e7cb2dfd14d21e72582 in term 1.
I20260809 15:08:01.122197 3635 leader_election.cc:302] T 655a85efa6c44eecb2bc8ab1aea4c8ba P cfc40b3492174e7cb2dfd14d21e72582 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ed67a7ab72e4760917f53986f1a774a, cfc40b3492174e7cb2dfd14d21e72582; no voters:
I20260809 15:08:01.122463 4498 raft_consensus.cc:2781] T 655a85efa6c44eecb2bc8ab1aea4c8ba P cfc40b3492174e7cb2dfd14d21e72582 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:08:01.122929 4498 raft_consensus.cc:686] T 655a85efa6c44eecb2bc8ab1aea4c8ba P cfc40b3492174e7cb2dfd14d21e72582 [term 1 LEADER]: Becoming Leader. State: Replica: cfc40b3492174e7cb2dfd14d21e72582, State: Running, Role: LEADER
I20260809 15:08:01.123215 4498 consensus_queue.cc:237] T 655a85efa6c44eecb2bc8ab1aea4c8ba P cfc40b3492174e7cb2dfd14d21e72582 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed67a7ab72e4760917f53986f1a774a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 40011 } } peers { permanent_uuid: "10025c6198174f61b5495c94ea3a510e" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46409 } } peers { permanent_uuid: "cfc40b3492174e7cb2dfd14d21e72582" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 44531 } }
I20260809 15:08:01.124014 4506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:01.123503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.24.183.193:60201)
0809 15:08:01.123657 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:01.123824 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 15:08:01.126066 3185 catalog_manager.cc:5495] T 655a85efa6c44eecb2bc8ab1aea4c8ba P cfc40b3492174e7cb2dfd14d21e72582 reported cstate change: term changed from 0 to 1, leader changed from <none> to cfc40b3492174e7cb2dfd14d21e72582 (127.24.183.193). New cstate: current_term: 1 leader_uuid: "cfc40b3492174e7cb2dfd14d21e72582" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed67a7ab72e4760917f53986f1a774a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 40011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10025c6198174f61b5495c94ea3a510e" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cfc40b3492174e7cb2dfd14d21e72582" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 44531 } health_report { overall_health: HEALTHY } } }
I20260809 15:08:01.143638 4481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:01.138945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37876 (local address 127.24.183.193:44531)
0809 15:08:01.139160 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:01.139166 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:01.139186 (+ 20us) server_negotiation.cc:407] Connection header received
0809 15:08:01.139249 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:01.139252 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:01.139317 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:01.139393 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:08:01.139833 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:01.140611 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:01.141987 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:01.142422 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:01.142559 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:01.142965 (+ 406us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:08:01.143080 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:08:01.143153 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:01.143300 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 15:08:01.143383 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 15:08:01.146716 4010 raft_consensus.cc:1261] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 10025c6198174f61b5495c94ea3a510e [term 1 FOLLOWER]: Refusing update from remote peer cfc40b3492174e7cb2dfd14d21e72582: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:08:01.146754 4236 raft_consensus.cc:1261] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 2ed67a7ab72e4760917f53986f1a774a [term 1 FOLLOWER]: Refusing update from remote peer cfc40b3492174e7cb2dfd14d21e72582: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:08:01.147342 4507 consensus_queue.cc:1035] T 655a85efa6c44eecb2bc8ab1aea4c8ba P cfc40b3492174e7cb2dfd14d21e72582 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ed67a7ab72e4760917f53986f1a774a" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 40011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:08:01.147710 4498 consensus_queue.cc:1035] T 655a85efa6c44eecb2bc8ab1aea4c8ba P cfc40b3492174e7cb2dfd14d21e72582 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10025c6198174f61b5495c94ea3a510e" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:08:01.150118 4504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:01.149605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.24.183.195:57605)
0809 15:08:01.149820 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:01.149921 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260809 15:08:01.150367 4505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:01.149572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.24.183.194:47211)
0809 15:08:01.149889 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:01.150091 (+ 202us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
I20260809 15:08:01.154336 4518 mvcc.cc:205] Tried to move back new op lower bound from 7316635980371165184 to 7316635980283461632. Current Snapshot: MvccSnapshot[applied={T|T < 7316635980371165184}]
I20260809 15:08:01.155969 4504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:01.155439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.24.183.195:56683)
0809 15:08:01.155704 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:01.155800 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
I20260809 15:08:01.159821 4522 mvcc.cc:205] Tried to move back new op lower bound from 7316635980371165184 to 7316635980283461632. Current Snapshot: MvccSnapshot[applied={T|T < 7316635980371165184}]
I20260809 15:08:01.164670 4505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:01.164194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.24.183.194:50989)
0809 15:08:01.164412 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:01.164502 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260809 15:08:01.165298 4506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:01.162923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.24.183.193:58881)
0809 15:08:01.163093 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:01.163186 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 15:08:01.166365 4506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:01.165893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.24.183.193:33021)
0809 15:08:01.166096 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:01.166185 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260809 15:08:01.173017 3185 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46878:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:08:01.173293 3185 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46878:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:08:01.178439 3185 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d835c269415b4f52b71dcbac3af1a081: Sending DeleteTablet for 3 replicas of tablet 655a85efa6c44eecb2bc8ab1aea4c8ba
I20260809 15:08:01.179450 4203 tablet_service.cc:1514] Processing DeleteTablet for tablet 655a85efa6c44eecb2bc8ab1aea4c8ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:08:01 UTC) from {username='oryx', principal='oryx/127.24.183.253@KRBTEST.COM'} at 127.0.0.1:36428
I20260809 15:08:01.179543 3977 tablet_service.cc:1514] Processing DeleteTablet for tablet 655a85efa6c44eecb2bc8ab1aea4c8ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:08:01 UTC) from {username='oryx', principal='oryx/127.24.183.253@KRBTEST.COM'} at 127.0.0.1:48576
I20260809 15:08:01.179579 3722 tablet_service.cc:1514] Processing DeleteTablet for tablet 655a85efa6c44eecb2bc8ab1aea4c8ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:08:01 UTC) from {username='oryx', principal='oryx/127.24.183.253@KRBTEST.COM'} at 127.0.0.1:37860
I20260809 15:08:01.180893 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 3478
I20260809 15:08:01.192075 4536 tablet_replica.cc:330] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 10025c6198174f61b5495c94ea3a510e: stopping tablet replica
I20260809 15:08:01.192267 4536 raft_consensus.cc:2229] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 10025c6198174f61b5495c94ea3a510e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:08:01.192564 4536 raft_consensus.cc:2258] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 10025c6198174f61b5495c94ea3a510e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:08:01.193830 4536 ts_tablet_manager.cc:1905] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 10025c6198174f61b5495c94ea3a510e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:08:01.194913 4534 tablet_replica.cc:330] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 2ed67a7ab72e4760917f53986f1a774a: stopping tablet replica
I20260809 15:08:01.195256 4534 raft_consensus.cc:2229] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 2ed67a7ab72e4760917f53986f1a774a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:08:01.195654 4534 raft_consensus.cc:2258] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 2ed67a7ab72e4760917f53986f1a774a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:08:01.196341 4535 tablet_replica.cc:330] T 655a85efa6c44eecb2bc8ab1aea4c8ba P cfc40b3492174e7cb2dfd14d21e72582: stopping tablet replica
I20260809 15:08:01.196595 4535 raft_consensus.cc:2229] T 655a85efa6c44eecb2bc8ab1aea4c8ba P cfc40b3492174e7cb2dfd14d21e72582 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:08:01.197095 4535 raft_consensus.cc:2258] T 655a85efa6c44eecb2bc8ab1aea4c8ba P cfc40b3492174e7cb2dfd14d21e72582 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:08:01.197168 4534 ts_tablet_manager.cc:1905] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 2ed67a7ab72e4760917f53986f1a774a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:08:01.198194 4535 ts_tablet_manager.cc:1905] T 655a85efa6c44eecb2bc8ab1aea4c8ba P cfc40b3492174e7cb2dfd14d21e72582: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:08:01.200539 4536 ts_tablet_manager.cc:1918] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 10025c6198174f61b5495c94ea3a510e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:08:01.200732 4536 log.cc:1196] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 10025c6198174f61b5495c94ea3a510e: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-1/wal/wals/655a85efa6c44eecb2bc8ab1aea4c8ba
I20260809 15:08:01.201260 4536 ts_tablet_manager.cc:1939] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 10025c6198174f61b5495c94ea3a510e: Deleting consensus metadata
I20260809 15:08:01.202543 3160 catalog_manager.cc:4841] TS 10025c6198174f61b5495c94ea3a510e (127.24.183.194:46409): tablet 655a85efa6c44eecb2bc8ab1aea4c8ba (table test-table [id=633be6ddb4c849a2a176ff260b2f62d9]) successfully deleted
I20260809 15:08:01.203914 4534 ts_tablet_manager.cc:1918] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 2ed67a7ab72e4760917f53986f1a774a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:08:01.204046 4534 log.cc:1196] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 2ed67a7ab72e4760917f53986f1a774a: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-2/wal/wals/655a85efa6c44eecb2bc8ab1aea4c8ba
I20260809 15:08:01.204501 4534 ts_tablet_manager.cc:1939] T 655a85efa6c44eecb2bc8ab1aea4c8ba P 2ed67a7ab72e4760917f53986f1a774a: Deleting consensus metadata
I20260809 15:08:01.204530 4535 ts_tablet_manager.cc:1918] T 655a85efa6c44eecb2bc8ab1aea4c8ba P cfc40b3492174e7cb2dfd14d21e72582: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:08:01.204674 4535 log.cc:1196] T 655a85efa6c44eecb2bc8ab1aea4c8ba P cfc40b3492174e7cb2dfd14d21e72582: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288021313853-25311-0/minicluster-data/ts-0/wal/wals/655a85efa6c44eecb2bc8ab1aea4c8ba
I20260809 15:08:01.205013 4535 ts_tablet_manager.cc:1939] T 655a85efa6c44eecb2bc8ab1aea4c8ba P cfc40b3492174e7cb2dfd14d21e72582: Deleting consensus metadata
I20260809 15:08:01.205633 3160 catalog_manager.cc:4841] TS 2ed67a7ab72e4760917f53986f1a774a (127.24.183.195:40011): tablet 655a85efa6c44eecb2bc8ab1aea4c8ba (table test-table [id=633be6ddb4c849a2a176ff260b2f62d9]) successfully deleted
I20260809 15:08:01.206018 3160 catalog_manager.cc:4841] TS cfc40b3492174e7cb2dfd14d21e72582 (127.24.183.193:44531): tablet 655a85efa6c44eecb2bc8ab1aea4c8ba (table test-table [id=633be6ddb4c849a2a176ff260b2f62d9]) successfully deleted
I20260809 15:08:01.206947 4481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:01.182621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37882 (local address 127.24.183.193:44531)
0809 15:08:01.182804 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:01.182811 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:01.188444 (+ 5633us) server_negotiation.cc:407] Connection header received
0809 15:08:01.188583 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:01.188589 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:01.188707 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:01.188815 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:01.190985 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:01.192300 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:01.194231 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:01.195036 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:01.195143 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:01.199236 (+ 4093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:01.199257 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:01.199283 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:08:01.199323 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:01.203507 (+ 4184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:01.204418 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:01.204427 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:01.204431 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:01.204641 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:01.205112 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:01.205118 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:01.205121 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:01.205697 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:01.206010 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:01.206543 (+ 533us) server_negotiation.cc:299] Negotiation successful
0809 15:08:01.206709 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 15:08:01.207551 3821 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:08:01.232661 4451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:01.232188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.0.0.1:37836)
0809 15:08:01.232396 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:01.232502 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 15:08:01.233808 4451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:01.233404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45957 (local address 127.0.0.1:37840)
0809 15:08:01.233604 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:01.233683 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:45957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260809 15:08:01.338953 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 3478
I20260809 15:08:01.370702 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 3856
I20260809 15:08:01.383358 4541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:01.371412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48580 (local address 127.24.183.194:46409)
0809 15:08:01.372118 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:01.372123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:01.372138 (+ 15us) server_negotiation.cc:407] Connection header received
0809 15:08:01.372192 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:01.372195 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:01.372264 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:01.372349 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:01.373627 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:01.374314 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:01.375459 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:01.375881 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:01.375981 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:01.378316 (+ 2335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:01.378333 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:01.378351 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:08:01.378394 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:01.380927 (+ 2533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:01.381562 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:01.381569 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:01.381571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:01.381709 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:01.382087 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:01.382090 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:01.382092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:01.382442 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:01.382675 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:01.382894 (+ 219us) server_negotiation.cc:299] Negotiation successful
0809 15:08:01.383075 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":502,"threads_started":1}
I20260809 15:08:01.384020 4077 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:08:01.478128 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 3856
I20260809 15:08:01.506110 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 4105
I20260809 15:08:01.519135 4548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:01.506869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36436 (local address 127.24.183.195:40011)
0809 15:08:01.507322 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:01.507327 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:01.507343 (+ 16us) server_negotiation.cc:407] Connection header received
0809 15:08:01.507408 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:01.507412 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:01.507479 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:01.507581 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:01.508951 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:01.509683 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:01.510904 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:01.511318 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:01.511438 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:01.513938 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:01.513953 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:01.513971 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:08:01.514004 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:01.516514 (+ 2510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:01.517159 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:01.517167 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:01.517169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:01.517290 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:01.517636 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:01.517639 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:01.517641 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:01.518017 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:01.518321 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:01.518591 (+ 270us) server_negotiation.cc:299] Negotiation successful
0809 15:08:01.518828 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":220,"threads_started":1}
I20260809 15:08:01.519840 4293 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:08:01.613386 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 4105
I20260809 15:08:01.639321 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 2969
I20260809 15:08:01.640479 3225 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:08:01.774606 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 2969
I20260809 15:08:01.804211 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 3127
I20260809 15:08:01.805318 3574 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:08:01.932969 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 3127
2026-08-09T15:08:01Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (9856 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:08:02 dist-test-slave-nj21 krb5kdc[4555](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:08:02 dist-test-slave-nj21 krb5kdc[4555](info): set up 2 sockets
Aug 09 15:08:02 dist-test-slave-nj21 krb5kdc[4555](info): commencing operation
krb5kdc: starting...
W20260809 15:08:03.188493 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.005s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:08:03 dist-test-slave-nj21 krb5kdc[4555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288083, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:08:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:08:03Z Disabled control of system clock
WARNING: no policy specified for oryx/127.24.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.254.keytab.
Entry for principal oryx/127.24.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.254.keytab.
I20260809 15:08:03.443068 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:34295
--webserver_interface=127.24.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.254.keytab
--principal=oryx/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:42981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:34295
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:08:03.769381 4571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:08:03.769685 4571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:08:03.769826 4571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:08:03.777189 4571 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:08:03.777299 4571 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 15:08:03.777371 4571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:08:03.777455 4571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:08:03.777518 4571 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:08:03.777601 4571 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:08:03.787500 4571 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:42981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:34295
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.254.keytab
--principal=oryx/127.24.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:34295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:08:03.788884 4571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:08:03.790277 4571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:08:03.801993 4586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:03.802030 4579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:03.801954 4580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:08:03.802971 4571 server_base.cc:1034] running on GCE node
I20260809 15:08:03.803764 4571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:08:03.805197 4571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:08:03.806420 4571 hybrid_clock.cc:648] HybridClock initialized: now 1786288083806358 us; error 37 us; skew 500 ppm
I20260809 15:08:03.806734 4571 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:08:03 dist-test-slave-nj21 krb5kdc[4555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288083, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:08:03.809880 4571 init.cc:377] Logged in from keytab as oryx/127.24.183.254@KRBTEST.COM (short username oryx)
I20260809 15:08:03.811327 4571 webserver.cc:460] Webserver started at http://127.24.183.254:37217/ using document root <none> and password file <none>
I20260809 15:08:03.812053 4571 fs_manager.cc:359] Metadata directory not provided
I20260809 15:08:03.812142 4571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:08:03.812433 4571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:08:03.815383 4571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "36c2e1a7410f4f569d153b1f1b6cc236"
format_stamp: "Formatted at 2026-08-09 15:08:03 on dist-test-slave-nj21"
server_key: "c028fd9ac743b1bc283a8312bfcda95c"
server_key_iv: "f9faf669a2feba6c5169a21e70ac088f"
server_key_version: "encryptionkey@0"
I20260809 15:08:03.816229 4571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "36c2e1a7410f4f569d153b1f1b6cc236"
format_stamp: "Formatted at 2026-08-09 15:08:03 on dist-test-slave-nj21"
server_key: "c028fd9ac743b1bc283a8312bfcda95c"
server_key_iv: "f9faf669a2feba6c5169a21e70ac088f"
server_key_version: "encryptionkey@0"
I20260809 15:08:03.821679 4571 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 15:08:03.825196 4600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:03.826511 4571 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 15:08:03.826676 4571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "36c2e1a7410f4f569d153b1f1b6cc236"
format_stamp: "Formatted at 2026-08-09 15:08:03 on dist-test-slave-nj21"
server_key: "c028fd9ac743b1bc283a8312bfcda95c"
server_key_iv: "f9faf669a2feba6c5169a21e70ac088f"
server_key_version: "encryptionkey@0"
I20260809 15:08:03.826876 4571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:08:03.847313 4571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:08:03.851393 4571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:08:03.851661 4571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:08:03.872378 4571 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:34295
I20260809 15:08:03.872398 4715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:34295 every 8 connection(s)
I20260809 15:08:03.873754 4571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:08:03.877556 4716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:08:03.882320 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 4571
I20260809 15:08:03.882664 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:08:03.883574 25311 external_mini_cluster.cc:1428] Setting key ea02d7b0ed699b960210a93895e78376
I20260809 15:08:03.889371 4716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36c2e1a7410f4f569d153b1f1b6cc236: Bootstrap starting.
I20260809 15:08:03.893111 4716 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36c2e1a7410f4f569d153b1f1b6cc236: Neither blocks nor log segments found. Creating new log.
I20260809 15:08:03.894102 4716 log.cc:824] T 00000000000000000000000000000000 P 36c2e1a7410f4f569d153b1f1b6cc236: Log is configured to *not* fsync() on all Append() calls
I20260809 15:08:03.896796 4716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36c2e1a7410f4f569d153b1f1b6cc236: No bootstrap required, opened a new log
Aug 09 15:08:03 dist-test-slave-nj21 krb5kdc[4555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288083, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM
I20260809 15:08:03.905534 4716 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36c2e1a7410f4f569d153b1f1b6cc236 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36c2e1a7410f4f569d153b1f1b6cc236" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 34295 } }
I20260809 15:08:03.905870 4716 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36c2e1a7410f4f569d153b1f1b6cc236 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:08:03.905968 4716 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36c2e1a7410f4f569d153b1f1b6cc236 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36c2e1a7410f4f569d153b1f1b6cc236, State: Initialized, Role: FOLLOWER
I20260809 15:08:03.906021 4721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:03.885672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48370 (local address 127.24.183.254:34295)
0809 15:08:03.886456 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:03.886468 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:03.886514 (+ 46us) server_negotiation.cc:407] Connection header received
0809 15:08:03.888295 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:03.888318 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:03.888668 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:03.888978 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:03.890809 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:03.892412 (+ 1603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:03.893810 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:03.894569 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:03.894770 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:03.898921 (+ 4151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:03.898931 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:03.898944 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:08:03.898984 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:03.901913 (+ 2929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:03.902681 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:03.902691 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:03.902696 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:03.902874 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:03.903265 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:03.903268 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:03.903270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:03.903865 (+ 595us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:03.904086 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:03.904607 (+ 521us) server_negotiation.cc:299] Negotiation successful
0809 15:08:03.905018 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":357,"threads_started":1}
I20260809 15:08:03.906520 4716 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36c2e1a7410f4f569d153b1f1b6cc236 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36c2e1a7410f4f569d153b1f1b6cc236" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 34295 } }
I20260809 15:08:03.906752 4716 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36c2e1a7410f4f569d153b1f1b6cc236 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:08:03.906888 4716 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36c2e1a7410f4f569d153b1f1b6cc236 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:08:03.907044 4716 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36c2e1a7410f4f569d153b1f1b6cc236 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:08:03.909914 4716 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36c2e1a7410f4f569d153b1f1b6cc236 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36c2e1a7410f4f569d153b1f1b6cc236" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 34295 } }
I20260809 15:08:03.910380 4716 leader_election.cc:302] T 00000000000000000000000000000000 P 36c2e1a7410f4f569d153b1f1b6cc236 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36c2e1a7410f4f569d153b1f1b6cc236; no voters:
I20260809 15:08:03.910921 4716 leader_election.cc:288] T 00000000000000000000000000000000 P 36c2e1a7410f4f569d153b1f1b6cc236 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:08:03.911042 4725 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36c2e1a7410f4f569d153b1f1b6cc236 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:08:03.911741 4725 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36c2e1a7410f4f569d153b1f1b6cc236 [term 1 LEADER]: Becoming Leader. State: Replica: 36c2e1a7410f4f569d153b1f1b6cc236, State: Running, Role: LEADER
I20260809 15:08:03.912278 4725 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36c2e1a7410f4f569d153b1f1b6cc236 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36c2e1a7410f4f569d153b1f1b6cc236" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 34295 } }
I20260809 15:08:03.912878 4716 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36c2e1a7410f4f569d153b1f1b6cc236 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:08:03.914839 4726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36c2e1a7410f4f569d153b1f1b6cc236 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36c2e1a7410f4f569d153b1f1b6cc236" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36c2e1a7410f4f569d153b1f1b6cc236" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 34295 } } }
I20260809 15:08:03.915186 4726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36c2e1a7410f4f569d153b1f1b6cc236 [sys.catalog]: This master's current role is: LEADER
I20260809 15:08:03.915648 4727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36c2e1a7410f4f569d153b1f1b6cc236 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36c2e1a7410f4f569d153b1f1b6cc236. Latest consensus state: current_term: 1 leader_uuid: "36c2e1a7410f4f569d153b1f1b6cc236" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36c2e1a7410f4f569d153b1f1b6cc236" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 34295 } } }
I20260809 15:08:03.916052 4727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36c2e1a7410f4f569d153b1f1b6cc236 [sys.catalog]: This master's current role is: LEADER
I20260809 15:08:03.917424 4730 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:08:03.923231 4730 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:08:03.932220 4730 catalog_manager.cc:1347] Generated new cluster ID: f919987867414fe5bdcce54a4199721a
I20260809 15:08:03.932335 4730 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:08:03.955948 4730 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:08:03.957315 4730 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:08:03.966168 4730 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36c2e1a7410f4f569d153b1f1b6cc236: Generated new TSK 0
I20260809 15:08:03.966907 4730 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.24.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.193.keytab.
Entry for principal oryx/127.24.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.193.keytab.
I20260809 15:08:04.028834 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.193:0
--local_ip_for_outbound_sockets=127.24.183.193
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:34295
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.193.keytab
--principal=oryx/127.24.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:42981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:08:04.366047 4754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:08:04.366341 4754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:08:04.366488 4754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:08:04.374056 4754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:08:04.374190 4754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:08:04.374323 4754 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 15:08:04.374454 4754 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.193
I20260809 15:08:04.384774 4754 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:42981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.193.keytab
--principal=oryx/127.24.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.183.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.183.254:34295
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.193
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:08:04.386097 4754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:08:04.387452 4754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:08:04.400197 4765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:04.400110 4769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:04.400853 4766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:08:04.402150 4754 server_base.cc:1034] running on GCE node
I20260809 15:08:04.402886 4754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:08:04.404102 4754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:08:04.405361 4754 hybrid_clock.cc:648] HybridClock initialized: now 1786288084405269 us; error 53 us; skew 500 ppm
I20260809 15:08:04.405728 4754 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:08:04 dist-test-slave-nj21 krb5kdc[4555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288084, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:08:04.410136 4754 init.cc:377] Logged in from keytab as oryx/127.24.183.193@KRBTEST.COM (short username oryx)
I20260809 15:08:04.413419 4754 webserver.cc:460] Webserver started at http://127.24.183.193:33545/ using document root <none> and password file <none>
I20260809 15:08:04.414366 4754 fs_manager.cc:359] Metadata directory not provided
I20260809 15:08:04.414486 4754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:08:04.414862 4754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:08:04.419061 4754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-0/data/instance:
uuid: "32c5ad23e4204495a9f41f9a7dd668ad"
format_stamp: "Formatted at 2026-08-09 15:08:04 on dist-test-slave-nj21"
server_key: "bde92d29edfd0f4fe319772ac15c8ea5"
server_key_iv: "4961456613fe52c515d75a0ff3b6a88f"
server_key_version: "encryptionkey@0"
I20260809 15:08:04.420146 4754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance:
uuid: "32c5ad23e4204495a9f41f9a7dd668ad"
format_stamp: "Formatted at 2026-08-09 15:08:04 on dist-test-slave-nj21"
server_key: "bde92d29edfd0f4fe319772ac15c8ea5"
server_key_iv: "4961456613fe52c515d75a0ff3b6a88f"
server_key_version: "encryptionkey@0"
I20260809 15:08:04.426996 4754 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260809 15:08:04.431245 4784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:04.432816 4754 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:08:04.432983 4754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-0/wal
uuid: "32c5ad23e4204495a9f41f9a7dd668ad"
format_stamp: "Formatted at 2026-08-09 15:08:04 on dist-test-slave-nj21"
server_key: "bde92d29edfd0f4fe319772ac15c8ea5"
server_key_iv: "4961456613fe52c515d75a0ff3b6a88f"
server_key_version: "encryptionkey@0"
I20260809 15:08:04.433187 4754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:08:04.458030 4754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:08:04.461771 4754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:08:04.462021 4754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:08:04.464777 4754 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:08:04.464879 4754 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:04.465011 4754 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:08:04.465090 4754 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 15:08:04 dist-test-slave-nj21 krb5kdc[4555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288084, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM
I20260809 15:08:04.487659 4804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:04.465976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:34295 (local address 127.24.183.193:50799)
0809 15:08:04.466765 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:04.466801 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:04.468734 (+ 1933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:04.469449 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:04.469467 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:04.470225 (+ 758us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:04.471157 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:04.471183 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:04.472803 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:04.472813 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:04.474828 (+ 2015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:04.474843 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:04.475627 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:04.475635 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:04.475942 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:04.476627 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:04.476649 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:04.479068 (+ 2419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:08:04.482934 (+ 3866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:08:04.482943 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:08:04.482961 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:08:04.483593 (+ 632us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:08:04.484008 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:08:04.484013 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:08:04.484021 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:08:04.484339 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:08:04.485110 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:08:04.485121 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:08:04.485537 (+ 416us) client_negotiation.cc:769] Sending connection context
0809 15:08:04.485883 (+ 346us) client_negotiation.cc:240] Negotiation successful
0809 15:08:04.486383 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":358,"threads_started":1}
I20260809 15:08:04.488209 4803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:04.465406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:50799 (local address 127.24.183.254:34295)
0809 15:08:04.466213 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:04.466219 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:04.466915 (+ 696us) server_negotiation.cc:407] Connection header received
0809 15:08:04.469005 (+ 2090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:04.469011 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:04.469101 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:04.469196 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:04.471438 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:04.472629 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:04.474974 (+ 2345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:04.475390 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:04.475505 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:04.479299 (+ 3794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:04.479304 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:04.479308 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:08:04.479342 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:04.482760 (+ 3418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:04.483731 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:04.483737 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:04.483741 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:04.483890 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:04.484553 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:04.484559 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:04.484562 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:04.484953 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:04.485108 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:04.487642 (+ 2534us) server_negotiation.cc:299] Negotiation successful
0809 15:08:04.487898 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":695,"thread_start_us":601,"threads_started":1}
I20260809 15:08:04.506294 4923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:04.494734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:34295 (local address 127.24.183.193:36185)
0809 15:08:04.495280 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:04.495295 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:04.495432 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:04.495755 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:04.495760 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:04.496187 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:04.496468 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:04.496476 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:04.497414 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:04.497420 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:04.498419 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:04.498430 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:04.499146 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:04.499153 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:04.499307 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:04.499971 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:04.499986 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:04.500670 (+ 684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:08:04.503721 (+ 3051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:08:04.503726 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:08:04.503729 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:08:04.504261 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:08:04.504733 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:08:04.504738 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:08:04.504742 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:08:04.504885 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:08:04.505483 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:08:04.505486 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:08:04.505631 (+ 145us) client_negotiation.cc:769] Sending connection context
0809 15:08:04.505734 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 15:08:04.505970 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":304,"threads_started":1}
I20260809 15:08:04.506488 4803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:04.494654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:36185 (local address 127.24.183.254:34295)
0809 15:08:04.494794 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:04.494798 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:04.495384 (+ 586us) server_negotiation.cc:407] Connection header received
0809 15:08:04.495555 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:04.495558 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:04.495616 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:04.495718 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:04.496600 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:04.497284 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:04.498548 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:04.498958 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:04.499065 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:04.500850 (+ 1785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:04.500857 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:04.500862 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:08:04.500901 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:04.503583 (+ 2682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:04.504428 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:04.504434 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:04.504438 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:04.504603 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:04.505022 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:04.505028 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:04.505031 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:04.505378 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:04.505549 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:04.505887 (+ 338us) server_negotiation.cc:299] Negotiation successful
0809 15:08:04.506136 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 15:08:04.513635 4754 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.193:39465
I20260809 15:08:04.513700 4978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.193:39465 every 8 connection(s)
I20260809 15:08:04.514789 4754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
I20260809 15:08:04.516671 4979 heartbeater.cc:347] Connected to a master server at 127.24.183.254:34295
I20260809 15:08:04.516885 4979 heartbeater.cc:464] Registering TS with master...
I20260809 15:08:04.517480 4979 heartbeater.cc:511] Master 127.24.183.254:34295 requested a full tablet report, sending...
I20260809 15:08:04.519369 4628 ts_manager.cc:194] Registered new tserver with Master: 32c5ad23e4204495a9f41f9a7dd668ad (127.24.183.193:39465)
I20260809 15:08:04.520790 4628 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.183.193@KRBTEST.COM'} at 127.24.183.193:50799
I20260809 15:08:04.523998 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 4754
I20260809 15:08:04.524151 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance
I20260809 15:08:04.524734 25311 external_mini_cluster.cc:1428] Setting key 97c30703c7d72565c9335d00eb76a48f
WARNING: no policy specified for oryx/127.24.183.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.183.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.194.keytab.
Entry for principal oryx/127.24.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.194.keytab.
I20260809 15:08:04.574203 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.194:0
--local_ip_for_outbound_sockets=127.24.183.194
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:34295
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.194.keytab
--principal=oryx/127.24.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:42981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:08:04.901069 4984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:08:04.901338 4984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:08:04.901456 4984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:08:04.908532 4984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:08:04.908635 4984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:08:04.908733 4984 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 15:08:04.908814 4984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.194
I20260809 15:08:04.918459 4984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:42981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.194.keytab
--principal=oryx/127.24.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.183.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.183.254:34295
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.194
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:08:04.919764 4984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:08:04.921028 4984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:08:04.933939 4994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:04.934019 5002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:04.934059 4993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:08:04.934930 4984 server_base.cc:1034] running on GCE node
I20260809 15:08:04.935601 4984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:08:04.936589 4984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:08:04.937860 4984 hybrid_clock.cc:648] HybridClock initialized: now 1786288084937769 us; error 52 us; skew 500 ppm
I20260809 15:08:04.938205 4984 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:08:04 dist-test-slave-nj21 krb5kdc[4555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288084, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:08:04.941537 4984 init.cc:377] Logged in from keytab as oryx/127.24.183.194@KRBTEST.COM (short username oryx)
I20260809 15:08:04.943205 4984 webserver.cc:460] Webserver started at http://127.24.183.194:46733/ using document root <none> and password file <none>
I20260809 15:08:04.944002 4984 fs_manager.cc:359] Metadata directory not provided
I20260809 15:08:04.944116 4984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:08:04.944468 4984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:08:04.947508 4984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-1/data/instance:
uuid: "e17f925643d44003bf98d29f94a48a4b"
format_stamp: "Formatted at 2026-08-09 15:08:04 on dist-test-slave-nj21"
server_key: "4bbb1214faa6a2c52d41d4d11e3278b0"
server_key_iv: "cc2423c1d30dc5deee779f65ab17b077"
server_key_version: "encryptionkey@0"
I20260809 15:08:04.948505 4984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance:
uuid: "e17f925643d44003bf98d29f94a48a4b"
format_stamp: "Formatted at 2026-08-09 15:08:04 on dist-test-slave-nj21"
server_key: "4bbb1214faa6a2c52d41d4d11e3278b0"
server_key_iv: "cc2423c1d30dc5deee779f65ab17b077"
server_key_version: "encryptionkey@0"
I20260809 15:08:04.954074 4984 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260809 15:08:04.957540 5011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:04.958843 4984 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:08:04.958994 4984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-1/wal
uuid: "e17f925643d44003bf98d29f94a48a4b"
format_stamp: "Formatted at 2026-08-09 15:08:04 on dist-test-slave-nj21"
server_key: "4bbb1214faa6a2c52d41d4d11e3278b0"
server_key_iv: "cc2423c1d30dc5deee779f65ab17b077"
server_key_version: "encryptionkey@0"
I20260809 15:08:04.959205 4984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:08:04.980765 4984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:08:04.984647 4984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:08:04.984927 4984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:08:04.987739 4984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:08:04.987830 4984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:04.987963 4984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:08:04.988022 4984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 15:08:04 dist-test-slave-nj21 krb5kdc[4555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288084, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM
I20260809 15:08:05.012574 4803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:04.988510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:49715 (local address 127.24.183.254:34295)
0809 15:08:04.988708 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:04.988715 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:04.990282 (+ 1567us) server_negotiation.cc:407] Connection header received
0809 15:08:04.993164 (+ 2882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:04.993172 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:04.993276 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:04.993416 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:04.995768 (+ 2352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:04.996739 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:04.999381 (+ 2642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.000127 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.000282 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:05.004777 (+ 4495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:05.004783 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:05.004787 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:08:05.004822 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:05.008271 (+ 3449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:05.009300 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:05.009306 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:05.009309 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:05.009479 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:05.010069 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:05.010074 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:05.010077 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:05.010447 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:05.010575 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:05.011875 (+ 1300us) server_negotiation.cc:299] Negotiation successful
0809 15:08:05.012127 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 15:08:05.014065 5027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:04.989098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:34295 (local address 127.24.183.194:49715)
0809 15:08:04.990037 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:04.990105 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:04.992742 (+ 2637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:04.993701 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:04.993714 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:04.994572 (+ 858us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:04.995489 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:04.995524 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:04.997011 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:04.997016 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:04.999200 (+ 2184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:04.999219 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.000371 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.000377 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:05.000565 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:05.001258 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:05.001287 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:05.004544 (+ 3257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:08:05.008423 (+ 3879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:08:05.008433 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:08:05.008454 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:08:05.009145 (+ 691us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:08:05.009601 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:08:05.009608 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:08:05.009612 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:08:05.009929 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:08:05.010578 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:08:05.010588 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:08:05.011111 (+ 523us) client_negotiation.cc:769] Sending connection context
0809 15:08:05.011480 (+ 369us) client_negotiation.cc:240] Negotiation successful
0809 15:08:05.012122 (+ 642us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":382,"threads_started":1}
I20260809 15:08:05.027510 4984 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.194:46433
I20260809 15:08:05.027555 5217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.194:46433 every 8 connection(s)
I20260809 15:08:05.028988 4984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
I20260809 15:08:05.031504 5218 heartbeater.cc:347] Connected to a master server at 127.24.183.254:34295
I20260809 15:08:05.031819 5218 heartbeater.cc:464] Registering TS with master...
I20260809 15:08:05.032722 5218 heartbeater.cc:511] Master 127.24.183.254:34295 requested a full tablet report, sending...
I20260809 15:08:05.034394 4628 ts_manager.cc:194] Registered new tserver with Master: e17f925643d44003bf98d29f94a48a4b (127.24.183.194:46433)
I20260809 15:08:05.034953 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 4984
I20260809 15:08:05.035125 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance
I20260809 15:08:05.035594 4628 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.183.194@KRBTEST.COM'} at 127.24.183.194:49715
I20260809 15:08:05.035710 25311 external_mini_cluster.cc:1428] Setting key 6191383ed08c88ef076bfefb3418529a
I20260809 15:08:05.036343 4803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:05.020984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:60491 (local address 127.24.183.254:34295)
0809 15:08:05.021187 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:05.021193 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:05.021785 (+ 592us) server_negotiation.cc:407] Connection header received
0809 15:08:05.022013 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:05.022020 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:05.022106 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:05.022224 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:05.023614 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.024944 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.026629 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.027349 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.027479 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:05.029597 (+ 2118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:05.029604 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:05.029609 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:08:05.029649 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:05.033456 (+ 3807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:05.034163 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:05.034168 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:05.034172 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:05.034351 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:05.034712 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:05.034718 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:05.034721 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:05.035150 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:05.035346 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:05.035767 (+ 421us) server_negotiation.cc:299] Negotiation successful
0809 15:08:05.035978 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":57}
I20260809 15:08:05.036433 5197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:05.021009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:34295 (local address 127.24.183.194:60491)
0809 15:08:05.021680 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:05.021700 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:05.021869 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:05.022272 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:05.022278 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:05.022955 (+ 677us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:05.023435 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:05.023449 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.025133 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.025139 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:05.026456 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:05.026474 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.027563 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.027570 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:05.027796 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:05.028446 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:05.028465 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:05.029340 (+ 875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:08:05.033594 (+ 4254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:08:05.033600 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:08:05.033604 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:08:05.034060 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:08:05.034450 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:08:05.034453 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:08:05.034456 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:08:05.034606 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:08:05.035316 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:08:05.035321 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:08:05.035498 (+ 177us) client_negotiation.cc:769] Sending connection context
0809 15:08:05.035624 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 15:08:05.035979 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":328,"threads_started":1}
WARNING: no policy specified for oryx/127.24.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.195.keytab.
Entry for principal oryx/127.24.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.195.keytab.
I20260809 15:08:05.087812 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.195:0
--local_ip_for_outbound_sockets=127.24.183.195
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:34295
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.195.keytab
--principal=oryx/127.24.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:42981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:08:05.419380 5226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:08:05.419627 5226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:08:05.419770 5226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:08:05.426748 5226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:08:05.426847 5226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:08:05.426937 5226 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 15:08:05.427014 5226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.195
I20260809 15:08:05.436578 5226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:42981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.195.keytab
--principal=oryx/127.24.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.183.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.183.254:34295
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.195
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:08:05.437726 5226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:08:05.439066 5226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:08:05.451557 5233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:05.451802 5234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:05.451804 5240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:08:05.452667 5226 server_base.cc:1034] running on GCE node
I20260809 15:08:05.453281 5226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:08:05.454113 5226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:08:05.455322 5226 hybrid_clock.cc:648] HybridClock initialized: now 1786288085455244 us; error 49 us; skew 500 ppm
I20260809 15:08:05.455586 5226 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:08:05 dist-test-slave-nj21 krb5kdc[4555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288085, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:08:05.458680 5226 init.cc:377] Logged in from keytab as oryx/127.24.183.195@KRBTEST.COM (short username oryx)
I20260809 15:08:05.460021 5226 webserver.cc:460] Webserver started at http://127.24.183.195:44585/ using document root <none> and password file <none>
I20260809 15:08:05.460651 5226 fs_manager.cc:359] Metadata directory not provided
I20260809 15:08:05.460732 5226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:08:05.460990 5226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:08:05.463735 5226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-2/data/instance:
uuid: "5565ec2abbf846858fef0f8311f80171"
format_stamp: "Formatted at 2026-08-09 15:08:05 on dist-test-slave-nj21"
server_key: "6109fda79b891125462c292dfad5ed97"
server_key_iv: "ee41f7dc41b6ed070260a476312e176b"
server_key_version: "encryptionkey@0"
I20260809 15:08:05.464478 5226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance:
uuid: "5565ec2abbf846858fef0f8311f80171"
format_stamp: "Formatted at 2026-08-09 15:08:05 on dist-test-slave-nj21"
server_key: "6109fda79b891125462c292dfad5ed97"
server_key_iv: "ee41f7dc41b6ed070260a476312e176b"
server_key_version: "encryptionkey@0"
I20260809 15:08:05.469208 5226 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 15:08:05.472432 5251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:05.473680 5226 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:08:05.473811 5226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-2/wal
uuid: "5565ec2abbf846858fef0f8311f80171"
format_stamp: "Formatted at 2026-08-09 15:08:05 on dist-test-slave-nj21"
server_key: "6109fda79b891125462c292dfad5ed97"
server_key_iv: "ee41f7dc41b6ed070260a476312e176b"
server_key_version: "encryptionkey@0"
I20260809 15:08:05.473980 5226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:08:05.490361 5226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:08:05.494248 5226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:08:05.494470 5226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:08:05.497102 5226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:08:05.497208 5226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:05.497356 5226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:08:05.497426 5226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 15:08:05 dist-test-slave-nj21 krb5kdc[4555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288085, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM
I20260809 15:08:05.519057 4803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:05.497460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:50033 (local address 127.24.183.254:34295)
0809 15:08:05.497622 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:05.497628 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:05.498967 (+ 1339us) server_negotiation.cc:407] Connection header received
0809 15:08:05.501036 (+ 2069us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:05.501042 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:05.501124 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:05.501215 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:05.503175 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.504386 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.506754 (+ 2368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.507412 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.507546 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:05.511906 (+ 4360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:05.511914 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:05.511919 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:08:05.511973 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:05.515520 (+ 3547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:05.516435 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:05.516439 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:05.516441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:05.516597 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:05.517178 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:05.517184 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:05.517187 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:05.517519 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:05.517697 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:05.518403 (+ 706us) server_negotiation.cc:299] Negotiation successful
0809 15:08:05.518651 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 15:08:05.520046 5269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:05.498032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:34295 (local address 127.24.183.195:50033)
0809 15:08:05.498804 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:05.498850 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:05.500749 (+ 1899us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:05.501472 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:05.501480 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:05.502155 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:05.502938 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:05.502979 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.504583 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.504589 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:05.506567 (+ 1978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:05.506579 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.507721 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.507730 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:05.508081 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:05.508736 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:05.508757 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:05.511612 (+ 2855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:08:05.515710 (+ 4098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:08:05.515724 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:08:05.515741 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:08:05.516278 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:08:05.516830 (+ 552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:08:05.516835 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:08:05.516839 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:08:05.517066 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:08:05.517646 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:08:05.517654 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:08:05.518004 (+ 350us) client_negotiation.cc:769] Sending connection context
0809 15:08:05.518234 (+ 230us) client_negotiation.cc:240] Negotiation successful
0809 15:08:05.518724 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":321,"threads_started":1}
I20260809 15:08:05.523196 4979 heartbeater.cc:503] Master 127.24.183.254:34295 was elected leader, sending a full tablet report...
I20260809 15:08:05.544653 5389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:05.530534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:34295 (local address 127.24.183.195:58943)
0809 15:08:05.531147 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:05.531163 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:05.531289 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:05.531627 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:05.531632 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:05.532270 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:05.532722 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:05.532743 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.534155 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.534160 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:05.535623 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:05.535643 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.536703 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.536712 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:05.536852 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:05.537493 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:05.537510 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:05.538414 (+ 904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:08:05.542165 (+ 3751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:08:05.542169 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:08:05.542171 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:08:05.542554 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:08:05.542933 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:08:05.542936 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:08:05.542938 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:08:05.543088 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:08:05.543638 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:08:05.543642 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:08:05.543846 (+ 204us) client_negotiation.cc:769] Sending connection context
0809 15:08:05.543980 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 15:08:05.544217 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":310,"threads_started":1}
I20260809 15:08:05.544711 4803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:05.530452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:58943 (local address 127.24.183.254:34295)
0809 15:08:05.530629 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:05.530634 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:05.531241 (+ 607us) server_negotiation.cc:407] Connection header received
0809 15:08:05.531412 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:05.531417 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:05.531481 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:05.531557 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:05.532918 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.534013 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.535827 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.536469 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.536602 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:05.538605 (+ 2003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:05.538613 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:05.538618 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:08:05.538661 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:05.542042 (+ 3381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:05.542649 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:05.542653 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:05.542656 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:05.542827 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:05.543204 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:05.543208 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:05.543211 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:05.543527 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:05.543733 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:05.544097 (+ 364us) server_negotiation.cc:299] Negotiation successful
0809 15:08:05.544335 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 15:08:05.553674 5226 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.195:36621
I20260809 15:08:05.553717 5448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.195:36621 every 8 connection(s)
I20260809 15:08:05.554759 5226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
I20260809 15:08:05.556607 5449 heartbeater.cc:347] Connected to a master server at 127.24.183.254:34295
I20260809 15:08:05.556824 5449 heartbeater.cc:464] Registering TS with master...
I20260809 15:08:05.557456 5449 heartbeater.cc:511] Master 127.24.183.254:34295 requested a full tablet report, sending...
I20260809 15:08:05.558722 4628 ts_manager.cc:194] Registered new tserver with Master: 5565ec2abbf846858fef0f8311f80171 (127.24.183.195:36621)
I20260809 15:08:05.559504 4628 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.183.195@KRBTEST.COM'} at 127.24.183.195:50033
I20260809 15:08:05.560300 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 5226
I20260809 15:08:05.560491 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance
I20260809 15:08:05.561108 25311 external_mini_cluster.cc:1428] Setting key 4b23d78db1a33b0f6c060307d0ffc7bd
I20260809 15:08:05.564034 25311 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 15:08:05 dist-test-slave-nj21 krb5kdc[4555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288085, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 15:08:05 dist-test-slave-nj21 krb5kdc[4555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:05 dist-test-slave-nj21 krb5kdc[4555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 15:08:05.620427 4803 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:05.609817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35794 (local address 127.24.183.254:34295)
0809 15:08:05.610051 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:05.610057 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:05.610288 (+ 231us) server_negotiation.cc:407] Connection header received
0809 15:08:05.610478 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:05.610483 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:05.610550 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:05.610638 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:05.612072 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.612891 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.614219 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.614681 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.614793 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:05.620114 (+ 5321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35794: BlockingRecv error: recv got EOF from 127.0.0.1:35794 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 09 15:08:05 dist-test-slave-nj21 krb5kdc[4555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288085, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM
I20260809 15:08:05.636711 4803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:05.623457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35800 (local address 127.24.183.254:34295)
0809 15:08:05.623635 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:05.623641 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:05.623956 (+ 315us) server_negotiation.cc:407] Connection header received
0809 15:08:05.624097 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:05.624101 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:05.624178 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:05.624259 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:05.625425 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.626136 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.627407 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.627882 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.628004 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:05.631665 (+ 3661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:05.631693 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:05.631698 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:08:05.631739 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:05.634082 (+ 2343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:05.634742 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:05.634747 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:05.634749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:05.634879 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:05.635226 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:05.635230 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:05.635232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:05.635499 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:05.635807 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:05.636103 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 15:08:05.636341 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 15:08:05.640573 4628 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35800:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:08:05.647619 4628 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:08:05.681340 5488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:05.659383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:36621 (local address 127.0.0.1:60588)
0809 15:08:05.660158 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:05.660203 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:05.660379 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:05.672476 (+ 12097us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:05.672485 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:05.672520 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:08:05.672996 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:05.673011 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.675175 (+ 2164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.675181 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:05.677882 (+ 2701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:05.677898 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.680227 (+ 2329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.680233 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:05.680475 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:05.680534 (+ 59us) client_negotiation.cc:769] Sending connection context
0809 15:08:05.680682 (+ 148us) client_negotiation.cc:240] Negotiation successful
0809 15:08:05.680889 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"mutex_wait_us":1,"thread_start_us":297,"threads_started":1}
I20260809 15:08:05.681342 5489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:05.659819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60588 (local address 127.24.183.195:36621)
0809 15:08:05.671357 (+ 11538us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:05.671368 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:05.671407 (+ 39us) server_negotiation.cc:407] Connection header received
0809 15:08:05.671542 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:05.671550 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:05.671856 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:05.672088 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:08:05.673289 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.675015 (+ 1726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.678098 (+ 3083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.679661 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.679867 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:05.680061 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:05.680843 (+ 782us) server_negotiation.cc:299] Negotiation successful
0809 15:08:05.681021 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11333,"thread_start_us":11050,"threads_started":1}
I20260809 15:08:05.685122 5491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:05.660487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:39465 (local address 127.0.0.1:40770)
0809 15:08:05.673324 (+ 12837us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:05.673342 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:05.673451 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:05.675653 (+ 2202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:05.675658 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:05.675727 (+ 69us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:08:05.676362 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:05.676377 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.678578 (+ 2201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.678583 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:05.682161 (+ 3578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:05.682177 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.684274 (+ 2097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.684281 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:05.684442 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:05.684477 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 15:08:05.684558 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 15:08:05.684686 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12729,"mutex_wait_us":1315,"spinlock_wait_cycles":12672,"thread_start_us":13035,"threads_started":1}
I20260809 15:08:05.685889 5490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:05.660285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:46433 (local address 127.0.0.1:52834)
0809 15:08:05.670791 (+ 10506us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:05.670811 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:05.670944 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:05.675782 (+ 4838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:05.675792 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:05.675817 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:08:05.676517 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:05.676530 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.678588 (+ 2058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.678593 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:05.682574 (+ 3981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:05.682603 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.684847 (+ 2244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.684854 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:05.685019 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:05.685056 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 15:08:05.685136 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 15:08:05.685503 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10368,"mutex_wait_us":1218,"thread_start_us":10199,"threads_started":1}
I20260809 15:08:05.686012 5494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:05.660957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40770 (local address 127.24.183.193:39465)
0809 15:08:05.673873 (+ 12916us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:05.673882 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:05.673922 (+ 40us) server_negotiation.cc:407] Connection header received
0809 15:08:05.674024 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:05.674033 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:05.674264 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:05.675411 (+ 1147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:08:05.676646 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.678417 (+ 1771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.682349 (+ 3932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.683782 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.683965 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:05.684130 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:05.685566 (+ 1436us) server_negotiation.cc:299] Negotiation successful
0809 15:08:05.685731 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12726,"thread_start_us":13517,"threads_started":1}
I20260809 15:08:05.686105 5492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:05.660478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52834 (local address 127.24.183.194:46433)
0809 15:08:05.674852 (+ 14374us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:05.674860 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:05.674887 (+ 27us) server_negotiation.cc:407] Connection header received
0809 15:08:05.675069 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:05.675079 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:05.675315 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:05.675528 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:08:05.676759 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.678351 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.682827 (+ 4476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.684260 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.684438 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:05.685592 (+ 1154us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:05.685681 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 15:08:05.685848 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14218,"thread_start_us":14112,"threads_started":1}
I20260809 15:08:05.686739 5332 tablet_service.cc:1467] Processing CreateTablet for tablet 10fc45202e63482e820abdc456a917f2 (DEFAULT_TABLE table=test-table [id=54390090b84d4a2ab167201b6f0e1d23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:08:05.688153 5332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10fc45202e63482e820abdc456a917f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:08:05.689491 4871 tablet_service.cc:1467] Processing CreateTablet for tablet 10fc45202e63482e820abdc456a917f2 (DEFAULT_TABLE table=test-table [id=54390090b84d4a2ab167201b6f0e1d23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:08:05.689491 5098 tablet_service.cc:1467] Processing CreateTablet for tablet 10fc45202e63482e820abdc456a917f2 (DEFAULT_TABLE table=test-table [id=54390090b84d4a2ab167201b6f0e1d23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:08:05.690845 4871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10fc45202e63482e820abdc456a917f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:08:05.690845 5098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10fc45202e63482e820abdc456a917f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:08:05.700584 5495 tablet_bootstrap.cc:492] T 10fc45202e63482e820abdc456a917f2 P 5565ec2abbf846858fef0f8311f80171: Bootstrap starting.
I20260809 15:08:05.703542 5495 tablet_bootstrap.cc:654] T 10fc45202e63482e820abdc456a917f2 P 5565ec2abbf846858fef0f8311f80171: Neither blocks nor log segments found. Creating new log.
I20260809 15:08:05.704655 5495 log.cc:824] T 10fc45202e63482e820abdc456a917f2 P 5565ec2abbf846858fef0f8311f80171: Log is configured to *not* fsync() on all Append() calls
I20260809 15:08:05.705478 5496 tablet_bootstrap.cc:492] T 10fc45202e63482e820abdc456a917f2 P 32c5ad23e4204495a9f41f9a7dd668ad: Bootstrap starting.
I20260809 15:08:05.705904 5497 tablet_bootstrap.cc:492] T 10fc45202e63482e820abdc456a917f2 P e17f925643d44003bf98d29f94a48a4b: Bootstrap starting.
I20260809 15:08:05.709035 5495 tablet_bootstrap.cc:492] T 10fc45202e63482e820abdc456a917f2 P 5565ec2abbf846858fef0f8311f80171: No bootstrap required, opened a new log
I20260809 15:08:05.709048 5496 tablet_bootstrap.cc:654] T 10fc45202e63482e820abdc456a917f2 P 32c5ad23e4204495a9f41f9a7dd668ad: Neither blocks nor log segments found. Creating new log.
I20260809 15:08:05.709251 5497 tablet_bootstrap.cc:654] T 10fc45202e63482e820abdc456a917f2 P e17f925643d44003bf98d29f94a48a4b: Neither blocks nor log segments found. Creating new log.
I20260809 15:08:05.709316 5495 ts_tablet_manager.cc:1397] T 10fc45202e63482e820abdc456a917f2 P 5565ec2abbf846858fef0f8311f80171: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 15:08:05.710024 5496 log.cc:824] T 10fc45202e63482e820abdc456a917f2 P 32c5ad23e4204495a9f41f9a7dd668ad: Log is configured to *not* fsync() on all Append() calls
I20260809 15:08:05.710451 5497 log.cc:824] T 10fc45202e63482e820abdc456a917f2 P e17f925643d44003bf98d29f94a48a4b: Log is configured to *not* fsync() on all Append() calls
I20260809 15:08:05.712723 5496 tablet_bootstrap.cc:492] T 10fc45202e63482e820abdc456a917f2 P 32c5ad23e4204495a9f41f9a7dd668ad: No bootstrap required, opened a new log
I20260809 15:08:05.712962 5496 ts_tablet_manager.cc:1397] T 10fc45202e63482e820abdc456a917f2 P 32c5ad23e4204495a9f41f9a7dd668ad: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260809 15:08:05.714236 5497 tablet_bootstrap.cc:492] T 10fc45202e63482e820abdc456a917f2 P e17f925643d44003bf98d29f94a48a4b: No bootstrap required, opened a new log
I20260809 15:08:05.714502 5497 ts_tablet_manager.cc:1397] T 10fc45202e63482e820abdc456a917f2 P e17f925643d44003bf98d29f94a48a4b: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260809 15:08:05.719867 5495 raft_consensus.cc:348] T 10fc45202e63482e820abdc456a917f2 P 5565ec2abbf846858fef0f8311f80171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } } peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } }
I20260809 15:08:05.720295 5495 raft_consensus.cc:374] T 10fc45202e63482e820abdc456a917f2 P 5565ec2abbf846858fef0f8311f80171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:08:05.720398 5495 raft_consensus.cc:729] T 10fc45202e63482e820abdc456a917f2 P 5565ec2abbf846858fef0f8311f80171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5565ec2abbf846858fef0f8311f80171, State: Initialized, Role: FOLLOWER
I20260809 15:08:05.720249 5496 raft_consensus.cc:348] T 10fc45202e63482e820abdc456a917f2 P 32c5ad23e4204495a9f41f9a7dd668ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } } peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } }
I20260809 15:08:05.720633 5496 raft_consensus.cc:374] T 10fc45202e63482e820abdc456a917f2 P 32c5ad23e4204495a9f41f9a7dd668ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:08:05.720749 5496 raft_consensus.cc:729] T 10fc45202e63482e820abdc456a917f2 P 32c5ad23e4204495a9f41f9a7dd668ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32c5ad23e4204495a9f41f9a7dd668ad, State: Initialized, Role: FOLLOWER
I20260809 15:08:05.721140 5495 consensus_queue.cc:260] T 10fc45202e63482e820abdc456a917f2 P 5565ec2abbf846858fef0f8311f80171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } } peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } }
I20260809 15:08:05.721290 5496 consensus_queue.cc:260] T 10fc45202e63482e820abdc456a917f2 P 32c5ad23e4204495a9f41f9a7dd668ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } } peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } }
I20260809 15:08:05.722544 5449 heartbeater.cc:503] Master 127.24.183.254:34295 was elected leader, sending a full tablet report...
I20260809 15:08:05.723066 5495 ts_tablet_manager.cc:1428] T 10fc45202e63482e820abdc456a917f2 P 5565ec2abbf846858fef0f8311f80171: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260809 15:08:05.724594 5496 ts_tablet_manager.cc:1428] T 10fc45202e63482e820abdc456a917f2 P 32c5ad23e4204495a9f41f9a7dd668ad: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260809 15:08:05.724663 5497 raft_consensus.cc:348] T 10fc45202e63482e820abdc456a917f2 P e17f925643d44003bf98d29f94a48a4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } } peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } }
I20260809 15:08:05.725047 5497 raft_consensus.cc:374] T 10fc45202e63482e820abdc456a917f2 P e17f925643d44003bf98d29f94a48a4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:08:05.725154 5497 raft_consensus.cc:729] T 10fc45202e63482e820abdc456a917f2 P e17f925643d44003bf98d29f94a48a4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e17f925643d44003bf98d29f94a48a4b, State: Initialized, Role: FOLLOWER
I20260809 15:08:05.725659 5497 consensus_queue.cc:260] T 10fc45202e63482e820abdc456a917f2 P e17f925643d44003bf98d29f94a48a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } } peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } }
I20260809 15:08:05.726677 5218 heartbeater.cc:503] Master 127.24.183.254:34295 was elected leader, sending a full tablet report...
I20260809 15:08:05.727149 5497 ts_tablet_manager.cc:1428] T 10fc45202e63482e820abdc456a917f2 P e17f925643d44003bf98d29f94a48a4b: Time spent starting tablet: real 0.013s user 0.013s sys 0.001s
W20260809 15:08:05.767122 4980 tablet.cc:2369] T 10fc45202e63482e820abdc456a917f2 P 32c5ad23e4204495a9f41f9a7dd668ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:08:05.781337 5219 tablet.cc:2369] T 10fc45202e63482e820abdc456a917f2 P e17f925643d44003bf98d29f94a48a4b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:08:05.806274 5450 tablet.cc:2369] T 10fc45202e63482e820abdc456a917f2 P 5565ec2abbf846858fef0f8311f80171: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:08:05.895015 5506 raft_consensus.cc:482] T 10fc45202e63482e820abdc456a917f2 P 32c5ad23e4204495a9f41f9a7dd668ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:08:05.895351 5506 raft_consensus.cc:504] T 10fc45202e63482e820abdc456a917f2 P 32c5ad23e4204495a9f41f9a7dd668ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } } peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } }
I20260809 15:08:05.897106 5506 leader_election.cc:288] T 10fc45202e63482e820abdc456a917f2 P 32c5ad23e4204495a9f41f9a7dd668ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5565ec2abbf846858fef0f8311f80171 (127.24.183.195:36621), e17f925643d44003bf98d29f94a48a4b (127.24.183.194:46433)
I20260809 15:08:05.903995 5489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:05.897541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:53927 (local address 127.24.183.195:36621)
0809 15:08:05.897762 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:05.897769 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:05.898601 (+ 832us) server_negotiation.cc:407] Connection header received
0809 15:08:05.898827 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:05.898832 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:05.898895 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:05.898969 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:08:05.899613 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.900345 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.902164 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.903063 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.903218 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:05.903297 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:05.903594 (+ 297us) server_negotiation.cc:299] Negotiation successful
0809 15:08:05.903745 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 15:08:05.904067 5492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:05.897637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:39317 (local address 127.24.183.194:46433)
0809 15:08:05.897844 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:05.897850 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:05.898591 (+ 741us) server_negotiation.cc:407] Connection header received
0809 15:08:05.898781 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:05.898786 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:05.898852 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:05.898937 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:08:05.899549 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.900355 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.902194 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.903069 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.903219 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:05.903297 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:05.903639 (+ 342us) server_negotiation.cc:299] Negotiation successful
0809 15:08:05.903793 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 15:08:05.904353 5509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:05.897659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:46433 (local address 127.24.183.193:39317)
0809 15:08:05.898462 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:05.898481 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:05.898637 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:05.898965 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:05.898969 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:05.898994 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:08:05.899377 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:05.899390 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.900496 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.900501 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:05.902033 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:05.902046 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.903357 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.903362 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:05.903485 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:05.903508 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 15:08:05.903588 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 15:08:05.903732 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"mutex_wait_us":23,"thread_start_us":441,"threads_started":1}
I20260809 15:08:05.904536 5510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:05.897671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:36621 (local address 127.24.183.193:53927)
0809 15:08:05.898488 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:05.898503 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:05.898691 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:05.899001 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:05.899006 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:05.899028 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:08:05.899448 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:05.899457 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.900484 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.900487 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:05.902018 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:05.902031 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.903308 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.903314 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:05.903442 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:05.903468 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 15:08:05.903550 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 15:08:05.903700 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":713,"mutex_wait_us":4,"spinlock_wait_cycles":25856,"thread_start_us":445,"threads_started":1}
I20260809 15:08:05.905623 5140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10fc45202e63482e820abdc456a917f2" candidate_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17f925643d44003bf98d29f94a48a4b" is_pre_election: true
I20260809 15:08:05.905615 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10fc45202e63482e820abdc456a917f2" candidate_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5565ec2abbf846858fef0f8311f80171" is_pre_election: true
I20260809 15:08:05.906212 5140 raft_consensus.cc:2454] T 10fc45202e63482e820abdc456a917f2 P e17f925643d44003bf98d29f94a48a4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32c5ad23e4204495a9f41f9a7dd668ad in term 0.
I20260809 15:08:05.906212 5386 raft_consensus.cc:2454] T 10fc45202e63482e820abdc456a917f2 P 5565ec2abbf846858fef0f8311f80171 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32c5ad23e4204495a9f41f9a7dd668ad in term 0.
I20260809 15:08:05.906929 4790 leader_election.cc:302] T 10fc45202e63482e820abdc456a917f2 P 32c5ad23e4204495a9f41f9a7dd668ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32c5ad23e4204495a9f41f9a7dd668ad, 5565ec2abbf846858fef0f8311f80171; no voters:
I20260809 15:08:05.907362 5506 raft_consensus.cc:2781] T 10fc45202e63482e820abdc456a917f2 P 32c5ad23e4204495a9f41f9a7dd668ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:08:05.907505 5506 raft_consensus.cc:482] T 10fc45202e63482e820abdc456a917f2 P 32c5ad23e4204495a9f41f9a7dd668ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:08:05.907611 5506 raft_consensus.cc:3037] T 10fc45202e63482e820abdc456a917f2 P 32c5ad23e4204495a9f41f9a7dd668ad [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:08:05.909932 5506 raft_consensus.cc:504] T 10fc45202e63482e820abdc456a917f2 P 32c5ad23e4204495a9f41f9a7dd668ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } } peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } }
I20260809 15:08:05.910949 5506 leader_election.cc:288] T 10fc45202e63482e820abdc456a917f2 P 32c5ad23e4204495a9f41f9a7dd668ad [CANDIDATE]: Term 1 election: Requested vote from peers 5565ec2abbf846858fef0f8311f80171 (127.24.183.195:36621), e17f925643d44003bf98d29f94a48a4b (127.24.183.194:46433)
I20260809 15:08:05.911369 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10fc45202e63482e820abdc456a917f2" candidate_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5565ec2abbf846858fef0f8311f80171"
I20260809 15:08:05.911399 5140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10fc45202e63482e820abdc456a917f2" candidate_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17f925643d44003bf98d29f94a48a4b"
I20260809 15:08:05.911550 5386 raft_consensus.cc:3037] T 10fc45202e63482e820abdc456a917f2 P 5565ec2abbf846858fef0f8311f80171 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:08:05.911592 5140 raft_consensus.cc:3037] T 10fc45202e63482e820abdc456a917f2 P e17f925643d44003bf98d29f94a48a4b [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:08:05.914227 5140 raft_consensus.cc:2454] T 10fc45202e63482e820abdc456a917f2 P e17f925643d44003bf98d29f94a48a4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32c5ad23e4204495a9f41f9a7dd668ad in term 1.
I20260809 15:08:05.914276 5386 raft_consensus.cc:2454] T 10fc45202e63482e820abdc456a917f2 P 5565ec2abbf846858fef0f8311f80171 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32c5ad23e4204495a9f41f9a7dd668ad in term 1.
I20260809 15:08:05.914610 4788 leader_election.cc:302] T 10fc45202e63482e820abdc456a917f2 P 32c5ad23e4204495a9f41f9a7dd668ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32c5ad23e4204495a9f41f9a7dd668ad, e17f925643d44003bf98d29f94a48a4b; no voters:
I20260809 15:08:05.914943 5506 raft_consensus.cc:2781] T 10fc45202e63482e820abdc456a917f2 P 32c5ad23e4204495a9f41f9a7dd668ad [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:08:05.915360 5506 raft_consensus.cc:686] T 10fc45202e63482e820abdc456a917f2 P 32c5ad23e4204495a9f41f9a7dd668ad [term 1 LEADER]: Becoming Leader. State: Replica: 32c5ad23e4204495a9f41f9a7dd668ad, State: Running, Role: LEADER
I20260809 15:08:05.916102 5506 consensus_queue.cc:237] T 10fc45202e63482e820abdc456a917f2 P 32c5ad23e4204495a9f41f9a7dd668ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } } peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } }
I20260809 15:08:05.921299 4628 catalog_manager.cc:5495] T 10fc45202e63482e820abdc456a917f2 P 32c5ad23e4204495a9f41f9a7dd668ad reported cstate change: term changed from 0 to 1, leader changed from <none> to 32c5ad23e4204495a9f41f9a7dd668ad (127.24.183.193). New cstate: current_term: 1 leader_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } health_report { overall_health: UNKNOWN } } }
I20260809 15:08:05.982836 5494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:05.977625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40772 (local address 127.24.183.193:39465)
0809 15:08:05.977872 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:05.977877 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:05.977923 (+ 46us) server_negotiation.cc:407] Connection header received
0809 15:08:05.978176 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:05.978180 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:05.978274 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:05.978405 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:08:05.978731 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.979449 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.980796 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:05.981216 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:05.981319 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:05.981682 (+ 363us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:08:05.982258 (+ 576us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:08:05.982370 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:05.982492 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 15:08:05.982565 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 15:08:05.990958 5386 raft_consensus.cc:1261] T 10fc45202e63482e820abdc456a917f2 P 5565ec2abbf846858fef0f8311f80171 [term 1 FOLLOWER]: Refusing update from remote peer 32c5ad23e4204495a9f41f9a7dd668ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:08:05.991072 5140 raft_consensus.cc:1261] T 10fc45202e63482e820abdc456a917f2 P e17f925643d44003bf98d29f94a48a4b [term 1 FOLLOWER]: Refusing update from remote peer 32c5ad23e4204495a9f41f9a7dd668ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:08:05.991945 5512 consensus_queue.cc:1035] T 10fc45202e63482e820abdc456a917f2 P 32c5ad23e4204495a9f41f9a7dd668ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:08:05.992295 5506 consensus_queue.cc:1035] T 10fc45202e63482e820abdc456a917f2 P 32c5ad23e4204495a9f41f9a7dd668ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:08:06.002637 5515 mvcc.cc:205] Tried to move back new op lower bound from 7316636000209448960 to 7316635999918403584. Current Snapshot: MvccSnapshot[applied={T|T < 7316636000209448960}]
I20260809 15:08:06.017676 4628 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:08:06.017984 4628 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:08:06.022564 4628 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 36c2e1a7410f4f569d153b1f1b6cc236: Sending DeleteTablet for 3 replicas of tablet 10fc45202e63482e820abdc456a917f2
I20260809 15:08:06.023797 5098 tablet_service.cc:1514] Processing DeleteTablet for tablet 10fc45202e63482e820abdc456a917f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:08:06 UTC) from {username='oryx', principal='oryx/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:52834
I20260809 15:08:06.023875 5332 tablet_service.cc:1514] Processing DeleteTablet for tablet 10fc45202e63482e820abdc456a917f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:08:06 UTC) from {username='oryx', principal='oryx/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:60588
I20260809 15:08:06.024479 4871 tablet_service.cc:1514] Processing DeleteTablet for tablet 10fc45202e63482e820abdc456a917f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:08:06 UTC) from {username='oryx', principal='oryx/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:40770
I20260809 15:08:06.024972 5532 tablet_replica.cc:330] T 10fc45202e63482e820abdc456a917f2 P e17f925643d44003bf98d29f94a48a4b: stopping tablet replica
I20260809 15:08:06.025378 5535 tablet_replica.cc:330] T 10fc45202e63482e820abdc456a917f2 P 32c5ad23e4204495a9f41f9a7dd668ad: stopping tablet replica
I20260809 15:08:06.025323 4628 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35800:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 15:08:06.025450 5532 raft_consensus.cc:2229] T 10fc45202e63482e820abdc456a917f2 P e17f925643d44003bf98d29f94a48a4b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:08:06.025794 5535 raft_consensus.cc:2229] T 10fc45202e63482e820abdc456a917f2 P 32c5ad23e4204495a9f41f9a7dd668ad [term 1 LEADER]: Raft consensus shutting down.
W20260809 15:08:06.026000 4628 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:08:06.026384 5532 raft_consensus.cc:2258] T 10fc45202e63482e820abdc456a917f2 P e17f925643d44003bf98d29f94a48a4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:08:06.026489 5535 raft_consensus.cc:2258] T 10fc45202e63482e820abdc456a917f2 P 32c5ad23e4204495a9f41f9a7dd668ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:08:06.028465 5535 ts_tablet_manager.cc:1905] T 10fc45202e63482e820abdc456a917f2 P 32c5ad23e4204495a9f41f9a7dd668ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:08:06.029110 5532 ts_tablet_manager.cc:1905] T 10fc45202e63482e820abdc456a917f2 P e17f925643d44003bf98d29f94a48a4b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:08:06.036459 4871 tablet_service.cc:1467] Processing CreateTablet for tablet 798f421b267541b299e899058ebafba1 (DEFAULT_TABLE table=test-table [id=5eade2520f7e4443a267e32608f8b9f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:08:06.036495 5330 tablet_service.cc:1467] Processing CreateTablet for tablet 798f421b267541b299e899058ebafba1 (DEFAULT_TABLE table=test-table [id=5eade2520f7e4443a267e32608f8b9f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:08:06.037052 4871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 798f421b267541b299e899058ebafba1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:08:06.037066 5330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 798f421b267541b299e899058ebafba1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:08:06.037606 5535 ts_tablet_manager.cc:1918] T 10fc45202e63482e820abdc456a917f2 P 32c5ad23e4204495a9f41f9a7dd668ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:08:06.037819 5535 log.cc:1196] T 10fc45202e63482e820abdc456a917f2 P 32c5ad23e4204495a9f41f9a7dd668ad: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-0/wal/wals/10fc45202e63482e820abdc456a917f2
I20260809 15:08:06.038331 5535 ts_tablet_manager.cc:1939] T 10fc45202e63482e820abdc456a917f2 P 32c5ad23e4204495a9f41f9a7dd668ad: Deleting consensus metadata
I20260809 15:08:06.040439 4605 catalog_manager.cc:4841] TS 32c5ad23e4204495a9f41f9a7dd668ad (127.24.183.193:39465): tablet 10fc45202e63482e820abdc456a917f2 (table test-table [id=54390090b84d4a2ab167201b6f0e1d23]) successfully deleted
I20260809 15:08:06.040568 5098 tablet_service.cc:1467] Processing CreateTablet for tablet 798f421b267541b299e899058ebafba1 (DEFAULT_TABLE table=test-table [id=5eade2520f7e4443a267e32608f8b9f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:08:06.041164 5098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 798f421b267541b299e899058ebafba1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:08:06.043918 5495 tablet_bootstrap.cc:492] T 798f421b267541b299e899058ebafba1 P 5565ec2abbf846858fef0f8311f80171: Bootstrap starting.
I20260809 15:08:06.045564 5495 tablet_bootstrap.cc:654] T 798f421b267541b299e899058ebafba1 P 5565ec2abbf846858fef0f8311f80171: Neither blocks nor log segments found. Creating new log.
I20260809 15:08:06.046391 5496 tablet_bootstrap.cc:492] T 798f421b267541b299e899058ebafba1 P 32c5ad23e4204495a9f41f9a7dd668ad: Bootstrap starting.
I20260809 15:08:06.048185 5497 tablet_bootstrap.cc:492] T 798f421b267541b299e899058ebafba1 P e17f925643d44003bf98d29f94a48a4b: Bootstrap starting.
I20260809 15:08:06.049013 5496 tablet_bootstrap.cc:654] T 798f421b267541b299e899058ebafba1 P 32c5ad23e4204495a9f41f9a7dd668ad: Neither blocks nor log segments found. Creating new log.
I20260809 15:08:06.049695 5497 tablet_bootstrap.cc:654] T 798f421b267541b299e899058ebafba1 P e17f925643d44003bf98d29f94a48a4b: Neither blocks nor log segments found. Creating new log.
I20260809 15:08:06.051628 5533 tablet_replica.cc:330] T 10fc45202e63482e820abdc456a917f2 P 5565ec2abbf846858fef0f8311f80171: stopping tablet replica
I20260809 15:08:06.051971 5497 tablet_bootstrap.cc:492] T 798f421b267541b299e899058ebafba1 P e17f925643d44003bf98d29f94a48a4b: No bootstrap required, opened a new log
I20260809 15:08:06.052119 5497 ts_tablet_manager.cc:1397] T 798f421b267541b299e899058ebafba1 P e17f925643d44003bf98d29f94a48a4b: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 15:08:06.052167 5533 raft_consensus.cc:2229] T 10fc45202e63482e820abdc456a917f2 P 5565ec2abbf846858fef0f8311f80171 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:08:06.052664 5533 raft_consensus.cc:2258] T 10fc45202e63482e820abdc456a917f2 P 5565ec2abbf846858fef0f8311f80171 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:08:06.053243 5497 raft_consensus.cc:348] T 798f421b267541b299e899058ebafba1 P e17f925643d44003bf98d29f94a48a4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } }
I20260809 15:08:06.053754 5497 raft_consensus.cc:374] T 798f421b267541b299e899058ebafba1 P e17f925643d44003bf98d29f94a48a4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:08:06.053961 5497 raft_consensus.cc:729] T 798f421b267541b299e899058ebafba1 P e17f925643d44003bf98d29f94a48a4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e17f925643d44003bf98d29f94a48a4b, State: Initialized, Role: FOLLOWER
I20260809 15:08:06.054572 5497 consensus_queue.cc:260] T 798f421b267541b299e899058ebafba1 P e17f925643d44003bf98d29f94a48a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } }
I20260809 15:08:06.057504 5497 ts_tablet_manager.cc:1428] T 798f421b267541b299e899058ebafba1 P e17f925643d44003bf98d29f94a48a4b: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260809 15:08:06.059321 5533 ts_tablet_manager.cc:1905] T 10fc45202e63482e820abdc456a917f2 P 5565ec2abbf846858fef0f8311f80171: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:08:06.062047 5496 tablet_bootstrap.cc:492] T 798f421b267541b299e899058ebafba1 P 32c5ad23e4204495a9f41f9a7dd668ad: No bootstrap required, opened a new log
I20260809 15:08:06.062269 5496 ts_tablet_manager.cc:1397] T 798f421b267541b299e899058ebafba1 P 32c5ad23e4204495a9f41f9a7dd668ad: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.010s
I20260809 15:08:06.063326 5496 raft_consensus.cc:348] T 798f421b267541b299e899058ebafba1 P 32c5ad23e4204495a9f41f9a7dd668ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } }
I20260809 15:08:06.063743 5496 raft_consensus.cc:374] T 798f421b267541b299e899058ebafba1 P 32c5ad23e4204495a9f41f9a7dd668ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:08:06.063925 5496 raft_consensus.cc:729] T 798f421b267541b299e899058ebafba1 P 32c5ad23e4204495a9f41f9a7dd668ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32c5ad23e4204495a9f41f9a7dd668ad, State: Initialized, Role: FOLLOWER
I20260809 15:08:06.064312 5496 consensus_queue.cc:260] T 798f421b267541b299e899058ebafba1 P 32c5ad23e4204495a9f41f9a7dd668ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } }
I20260809 15:08:06.065303 5532 ts_tablet_manager.cc:1918] T 10fc45202e63482e820abdc456a917f2 P e17f925643d44003bf98d29f94a48a4b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:08:06.065608 5532 log.cc:1196] T 10fc45202e63482e820abdc456a917f2 P e17f925643d44003bf98d29f94a48a4b: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-1/wal/wals/10fc45202e63482e820abdc456a917f2
I20260809 15:08:06.066561 5496 ts_tablet_manager.cc:1428] T 798f421b267541b299e899058ebafba1 P 32c5ad23e4204495a9f41f9a7dd668ad: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260809 15:08:06.066633 5532 ts_tablet_manager.cc:1939] T 10fc45202e63482e820abdc456a917f2 P e17f925643d44003bf98d29f94a48a4b: Deleting consensus metadata
I20260809 15:08:06.069141 4603 catalog_manager.cc:4841] TS e17f925643d44003bf98d29f94a48a4b (127.24.183.194:46433): tablet 10fc45202e63482e820abdc456a917f2 (table test-table [id=54390090b84d4a2ab167201b6f0e1d23]) successfully deleted
I20260809 15:08:06.069944 5495 tablet_bootstrap.cc:492] T 798f421b267541b299e899058ebafba1 P 5565ec2abbf846858fef0f8311f80171: No bootstrap required, opened a new log
I20260809 15:08:06.070086 5495 ts_tablet_manager.cc:1397] T 798f421b267541b299e899058ebafba1 P 5565ec2abbf846858fef0f8311f80171: Time spent bootstrapping tablet: real 0.026s user 0.007s sys 0.002s
I20260809 15:08:06.070829 5495 raft_consensus.cc:348] T 798f421b267541b299e899058ebafba1 P 5565ec2abbf846858fef0f8311f80171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } }
I20260809 15:08:06.071038 5495 raft_consensus.cc:374] T 798f421b267541b299e899058ebafba1 P 5565ec2abbf846858fef0f8311f80171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:08:06.071135 5495 raft_consensus.cc:729] T 798f421b267541b299e899058ebafba1 P 5565ec2abbf846858fef0f8311f80171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5565ec2abbf846858fef0f8311f80171, State: Initialized, Role: FOLLOWER
I20260809 15:08:06.071405 5495 consensus_queue.cc:260] T 798f421b267541b299e899058ebafba1 P 5565ec2abbf846858fef0f8311f80171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } }
I20260809 15:08:06.071955 5495 ts_tablet_manager.cc:1428] T 798f421b267541b299e899058ebafba1 P 5565ec2abbf846858fef0f8311f80171: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 15:08:06.074340 5533 ts_tablet_manager.cc:1918] T 10fc45202e63482e820abdc456a917f2 P 5565ec2abbf846858fef0f8311f80171: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:08:06.074501 5533 log.cc:1196] T 10fc45202e63482e820abdc456a917f2 P 5565ec2abbf846858fef0f8311f80171: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-2/wal/wals/10fc45202e63482e820abdc456a917f2
I20260809 15:08:06.074904 5533 ts_tablet_manager.cc:1939] T 10fc45202e63482e820abdc456a917f2 P 5565ec2abbf846858fef0f8311f80171: Deleting consensus metadata
I20260809 15:08:06.076422 4605 catalog_manager.cc:4841] TS 5565ec2abbf846858fef0f8311f80171 (127.24.183.195:36621): tablet 10fc45202e63482e820abdc456a917f2 (table test-table [id=54390090b84d4a2ab167201b6f0e1d23]) successfully deleted
I20260809 15:08:06.120527 5506 raft_consensus.cc:482] T 798f421b267541b299e899058ebafba1 P 32c5ad23e4204495a9f41f9a7dd668ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:08:06.120750 5506 raft_consensus.cc:504] T 798f421b267541b299e899058ebafba1 P 32c5ad23e4204495a9f41f9a7dd668ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } }
I20260809 15:08:06.121454 5506 leader_election.cc:288] T 798f421b267541b299e899058ebafba1 P 32c5ad23e4204495a9f41f9a7dd668ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e17f925643d44003bf98d29f94a48a4b (127.24.183.194:46433), 5565ec2abbf846858fef0f8311f80171 (127.24.183.195:36621)
I20260809 15:08:06.121992 5140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "798f421b267541b299e899058ebafba1" candidate_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17f925643d44003bf98d29f94a48a4b" is_pre_election: true
I20260809 15:08:06.122174 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "798f421b267541b299e899058ebafba1" candidate_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5565ec2abbf846858fef0f8311f80171" is_pre_election: true
I20260809 15:08:06.122299 5140 raft_consensus.cc:2454] T 798f421b267541b299e899058ebafba1 P e17f925643d44003bf98d29f94a48a4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32c5ad23e4204495a9f41f9a7dd668ad in term 0.
I20260809 15:08:06.122466 5386 raft_consensus.cc:2454] T 798f421b267541b299e899058ebafba1 P 5565ec2abbf846858fef0f8311f80171 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32c5ad23e4204495a9f41f9a7dd668ad in term 0.
I20260809 15:08:06.122778 4788 leader_election.cc:302] T 798f421b267541b299e899058ebafba1 P 32c5ad23e4204495a9f41f9a7dd668ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32c5ad23e4204495a9f41f9a7dd668ad, e17f925643d44003bf98d29f94a48a4b; no voters:
I20260809 15:08:06.123087 5506 raft_consensus.cc:2781] T 798f421b267541b299e899058ebafba1 P 32c5ad23e4204495a9f41f9a7dd668ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:08:06.123239 5506 raft_consensus.cc:482] T 798f421b267541b299e899058ebafba1 P 32c5ad23e4204495a9f41f9a7dd668ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:08:06.123346 5506 raft_consensus.cc:3037] T 798f421b267541b299e899058ebafba1 P 32c5ad23e4204495a9f41f9a7dd668ad [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:08:06.126585 5506 raft_consensus.cc:504] T 798f421b267541b299e899058ebafba1 P 32c5ad23e4204495a9f41f9a7dd668ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } }
I20260809 15:08:06.127224 5506 leader_election.cc:288] T 798f421b267541b299e899058ebafba1 P 32c5ad23e4204495a9f41f9a7dd668ad [CANDIDATE]: Term 1 election: Requested vote from peers e17f925643d44003bf98d29f94a48a4b (127.24.183.194:46433), 5565ec2abbf846858fef0f8311f80171 (127.24.183.195:36621)
I20260809 15:08:06.127761 5140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "798f421b267541b299e899058ebafba1" candidate_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17f925643d44003bf98d29f94a48a4b"
I20260809 15:08:06.127737 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "798f421b267541b299e899058ebafba1" candidate_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5565ec2abbf846858fef0f8311f80171"
I20260809 15:08:06.127941 5386 raft_consensus.cc:3037] T 798f421b267541b299e899058ebafba1 P 5565ec2abbf846858fef0f8311f80171 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:08:06.127945 5140 raft_consensus.cc:3037] T 798f421b267541b299e899058ebafba1 P e17f925643d44003bf98d29f94a48a4b [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:08:06.130182 5140 raft_consensus.cc:2454] T 798f421b267541b299e899058ebafba1 P e17f925643d44003bf98d29f94a48a4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32c5ad23e4204495a9f41f9a7dd668ad in term 1.
I20260809 15:08:06.130654 4788 leader_election.cc:302] T 798f421b267541b299e899058ebafba1 P 32c5ad23e4204495a9f41f9a7dd668ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32c5ad23e4204495a9f41f9a7dd668ad, e17f925643d44003bf98d29f94a48a4b; no voters:
I20260809 15:08:06.130944 5506 raft_consensus.cc:2781] T 798f421b267541b299e899058ebafba1 P 32c5ad23e4204495a9f41f9a7dd668ad [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:08:06.130986 5386 raft_consensus.cc:2454] T 798f421b267541b299e899058ebafba1 P 5565ec2abbf846858fef0f8311f80171 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32c5ad23e4204495a9f41f9a7dd668ad in term 1.
I20260809 15:08:06.131312 5506 raft_consensus.cc:686] T 798f421b267541b299e899058ebafba1 P 32c5ad23e4204495a9f41f9a7dd668ad [term 1 LEADER]: Becoming Leader. State: Replica: 32c5ad23e4204495a9f41f9a7dd668ad, State: Running, Role: LEADER
I20260809 15:08:06.131785 5506 consensus_queue.cc:237] T 798f421b267541b299e899058ebafba1 P 32c5ad23e4204495a9f41f9a7dd668ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } }
I20260809 15:08:06.134523 4627 catalog_manager.cc:5495] T 798f421b267541b299e899058ebafba1 P 32c5ad23e4204495a9f41f9a7dd668ad reported cstate change: term changed from 0 to 1, leader changed from <none> to 32c5ad23e4204495a9f41f9a7dd668ad (127.24.183.193). New cstate: current_term: 1 leader_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } health_report { overall_health: UNKNOWN } } }
Aug 09 15:08:06 dist-test-slave-nj21 krb5kdc[4555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288083, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.254@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM
I20260809 15:08:06.193431 5554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:06.174080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:34295 (local address 127.0.0.1:35802)
0809 15:08:06.174901 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:06.174921 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:06.175070 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:06.175405 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:06.175410 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:06.176413 (+ 1003us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:06.176792 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:06.176806 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:06.178022 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:06.178028 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:06.179585 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:06.179600 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:06.180402 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:06.180410 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:06.180583 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:06.184364 (+ 3781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:06.184389 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:06.187338 (+ 2949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:08:06.190712 (+ 3374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:08:06.190727 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:08:06.190736 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:08:06.191251 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:08:06.191623 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:08:06.191628 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:08:06.191633 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:08:06.191809 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:08:06.192341 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:08:06.192350 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:08:06.192540 (+ 190us) client_negotiation.cc:769] Sending connection context
0809 15:08:06.192682 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 15:08:06.192991 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"thread_start_us":522,"threads_started":1}
I20260809 15:08:06.193480 5553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:06.174088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35802 (local address 127.24.183.254:34295)
0809 15:08:06.174769 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:06.174775 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:06.175006 (+ 231us) server_negotiation.cc:407] Connection header received
0809 15:08:06.175188 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:06.175192 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:06.175253 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:06.175333 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:06.176957 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:06.177882 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:06.179779 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:06.180196 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:06.180291 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:06.187536 (+ 7245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:06.187541 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:06.187544 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:08:06.187575 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:06.190579 (+ 3004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:06.191369 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:06.191373 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:06.191375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:06.191518 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:06.191917 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:06.191920 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:06.191922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:06.192213 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:06.192348 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:06.192802 (+ 454us) server_negotiation.cc:299] Negotiation successful
0809 15:08:06.193123 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":126,"server-negotiator.queue_time_us":491,"thread_start_us":285,"threads_started":1}
I20260809 15:08:06.199290 4627 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:35802:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 15:08:06.200374 4627 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:08:06.207379 4871 tablet_service.cc:1467] Processing CreateTablet for tablet f4691e48b7ef4cbfb02901dfd12f84cb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cb20590e74f04327ae523ce5eb062a90]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 15:08:06.207799 5332 tablet_service.cc:1467] Processing CreateTablet for tablet f4691e48b7ef4cbfb02901dfd12f84cb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cb20590e74f04327ae523ce5eb062a90]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 15:08:06.207841 5098 tablet_service.cc:1467] Processing CreateTablet for tablet f4691e48b7ef4cbfb02901dfd12f84cb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cb20590e74f04327ae523ce5eb062a90]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 15:08:06.208113 4871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4691e48b7ef4cbfb02901dfd12f84cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:08:06.208385 5332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4691e48b7ef4cbfb02901dfd12f84cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:08:06.208436 5098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4691e48b7ef4cbfb02901dfd12f84cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:08:06.218119 5496 tablet_bootstrap.cc:492] T f4691e48b7ef4cbfb02901dfd12f84cb P 32c5ad23e4204495a9f41f9a7dd668ad: Bootstrap starting.
I20260809 15:08:06.218199 5495 tablet_bootstrap.cc:492] T f4691e48b7ef4cbfb02901dfd12f84cb P 5565ec2abbf846858fef0f8311f80171: Bootstrap starting.
I20260809 15:08:06.220371 5495 tablet_bootstrap.cc:654] T f4691e48b7ef4cbfb02901dfd12f84cb P 5565ec2abbf846858fef0f8311f80171: Neither blocks nor log segments found. Creating new log.
I20260809 15:08:06.220389 5496 tablet_bootstrap.cc:654] T f4691e48b7ef4cbfb02901dfd12f84cb P 32c5ad23e4204495a9f41f9a7dd668ad: Neither blocks nor log segments found. Creating new log.
I20260809 15:08:06.220932 5497 tablet_bootstrap.cc:492] T f4691e48b7ef4cbfb02901dfd12f84cb P e17f925643d44003bf98d29f94a48a4b: Bootstrap starting.
I20260809 15:08:06.222573 5497 tablet_bootstrap.cc:654] T f4691e48b7ef4cbfb02901dfd12f84cb P e17f925643d44003bf98d29f94a48a4b: Neither blocks nor log segments found. Creating new log.
I20260809 15:08:06.223100 5495 tablet_bootstrap.cc:492] T f4691e48b7ef4cbfb02901dfd12f84cb P 5565ec2abbf846858fef0f8311f80171: No bootstrap required, opened a new log
I20260809 15:08:06.223515 5495 ts_tablet_manager.cc:1397] T f4691e48b7ef4cbfb02901dfd12f84cb P 5565ec2abbf846858fef0f8311f80171: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260809 15:08:06.223949 5496 tablet_bootstrap.cc:492] T f4691e48b7ef4cbfb02901dfd12f84cb P 32c5ad23e4204495a9f41f9a7dd668ad: No bootstrap required, opened a new log
I20260809 15:08:06.224138 5496 ts_tablet_manager.cc:1397] T f4691e48b7ef4cbfb02901dfd12f84cb P 32c5ad23e4204495a9f41f9a7dd668ad: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260809 15:08:06.224570 5495 raft_consensus.cc:348] T f4691e48b7ef4cbfb02901dfd12f84cb P 5565ec2abbf846858fef0f8311f80171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } }
I20260809 15:08:06.224839 5495 raft_consensus.cc:374] T f4691e48b7ef4cbfb02901dfd12f84cb P 5565ec2abbf846858fef0f8311f80171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:08:06.224753 5496 raft_consensus.cc:348] T f4691e48b7ef4cbfb02901dfd12f84cb P 32c5ad23e4204495a9f41f9a7dd668ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } }
I20260809 15:08:06.224920 5495 raft_consensus.cc:729] T f4691e48b7ef4cbfb02901dfd12f84cb P 5565ec2abbf846858fef0f8311f80171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5565ec2abbf846858fef0f8311f80171, State: Initialized, Role: FOLLOWER
I20260809 15:08:06.224990 5496 raft_consensus.cc:374] T f4691e48b7ef4cbfb02901dfd12f84cb P 32c5ad23e4204495a9f41f9a7dd668ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:08:06.225070 5496 raft_consensus.cc:729] T f4691e48b7ef4cbfb02901dfd12f84cb P 32c5ad23e4204495a9f41f9a7dd668ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32c5ad23e4204495a9f41f9a7dd668ad, State: Initialized, Role: FOLLOWER
I20260809 15:08:06.225160 5495 consensus_queue.cc:260] T f4691e48b7ef4cbfb02901dfd12f84cb P 5565ec2abbf846858fef0f8311f80171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } }
I20260809 15:08:06.225304 5496 consensus_queue.cc:260] T f4691e48b7ef4cbfb02901dfd12f84cb P 32c5ad23e4204495a9f41f9a7dd668ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } }
I20260809 15:08:06.225628 5503 tablet_replica.cc:439] T f4691e48b7ef4cbfb02901dfd12f84cb P 5565ec2abbf846858fef0f8311f80171: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } } }
I20260809 15:08:06.225909 5496 ts_tablet_manager.cc:1428] T f4691e48b7ef4cbfb02901dfd12f84cb P 32c5ad23e4204495a9f41f9a7dd668ad: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 15:08:06.225946 5503 tablet_replica.cc:442] T f4691e48b7ef4cbfb02901dfd12f84cb P 5565ec2abbf846858fef0f8311f80171: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:08:06.225733 5506 tablet_replica.cc:439] T f4691e48b7ef4cbfb02901dfd12f84cb P 32c5ad23e4204495a9f41f9a7dd668ad: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } } }
I20260809 15:08:06.226116 5506 tablet_replica.cc:442] T f4691e48b7ef4cbfb02901dfd12f84cb P 32c5ad23e4204495a9f41f9a7dd668ad: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:08:06.227737 5495 ts_tablet_manager.cc:1428] T f4691e48b7ef4cbfb02901dfd12f84cb P 5565ec2abbf846858fef0f8311f80171: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260809 15:08:06.230130 5497 tablet_bootstrap.cc:492] T f4691e48b7ef4cbfb02901dfd12f84cb P e17f925643d44003bf98d29f94a48a4b: No bootstrap required, opened a new log
I20260809 15:08:06.230352 5497 ts_tablet_manager.cc:1397] T f4691e48b7ef4cbfb02901dfd12f84cb P e17f925643d44003bf98d29f94a48a4b: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.005s
I20260809 15:08:06.230947 5497 raft_consensus.cc:348] T f4691e48b7ef4cbfb02901dfd12f84cb P e17f925643d44003bf98d29f94a48a4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } }
I20260809 15:08:06.231135 5497 raft_consensus.cc:374] T f4691e48b7ef4cbfb02901dfd12f84cb P e17f925643d44003bf98d29f94a48a4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:08:06.231210 5497 raft_consensus.cc:729] T f4691e48b7ef4cbfb02901dfd12f84cb P e17f925643d44003bf98d29f94a48a4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e17f925643d44003bf98d29f94a48a4b, State: Initialized, Role: FOLLOWER
I20260809 15:08:06.231451 5497 consensus_queue.cc:260] T f4691e48b7ef4cbfb02901dfd12f84cb P e17f925643d44003bf98d29f94a48a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } }
I20260809 15:08:06.232100 5508 tablet_replica.cc:439] T f4691e48b7ef4cbfb02901dfd12f84cb P e17f925643d44003bf98d29f94a48a4b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } } }
I20260809 15:08:06.232390 5508 tablet_replica.cc:442] T f4691e48b7ef4cbfb02901dfd12f84cb P e17f925643d44003bf98d29f94a48a4b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:08:06.232615 5497 ts_tablet_manager.cc:1428] T f4691e48b7ef4cbfb02901dfd12f84cb P e17f925643d44003bf98d29f94a48a4b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 15:08:06.272801 5506 raft_consensus.cc:482] T f4691e48b7ef4cbfb02901dfd12f84cb P 32c5ad23e4204495a9f41f9a7dd668ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:08:06.273074 5506 raft_consensus.cc:504] T f4691e48b7ef4cbfb02901dfd12f84cb P 32c5ad23e4204495a9f41f9a7dd668ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } }
I20260809 15:08:06.273811 5506 leader_election.cc:288] T f4691e48b7ef4cbfb02901dfd12f84cb P 32c5ad23e4204495a9f41f9a7dd668ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e17f925643d44003bf98d29f94a48a4b (127.24.183.194:46433), 5565ec2abbf846858fef0f8311f80171 (127.24.183.195:36621)
I20260809 15:08:06.274339 5140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4691e48b7ef4cbfb02901dfd12f84cb" candidate_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17f925643d44003bf98d29f94a48a4b" is_pre_election: true
I20260809 15:08:06.274390 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4691e48b7ef4cbfb02901dfd12f84cb" candidate_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5565ec2abbf846858fef0f8311f80171" is_pre_election: true
I20260809 15:08:06.274588 5140 raft_consensus.cc:2454] T f4691e48b7ef4cbfb02901dfd12f84cb P e17f925643d44003bf98d29f94a48a4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32c5ad23e4204495a9f41f9a7dd668ad in term 0.
I20260809 15:08:06.274675 5386 raft_consensus.cc:2454] T f4691e48b7ef4cbfb02901dfd12f84cb P 5565ec2abbf846858fef0f8311f80171 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32c5ad23e4204495a9f41f9a7dd668ad in term 0.
I20260809 15:08:06.275039 4788 leader_election.cc:302] T f4691e48b7ef4cbfb02901dfd12f84cb P 32c5ad23e4204495a9f41f9a7dd668ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32c5ad23e4204495a9f41f9a7dd668ad, e17f925643d44003bf98d29f94a48a4b; no voters:
I20260809 15:08:06.275359 5506 raft_consensus.cc:2781] T f4691e48b7ef4cbfb02901dfd12f84cb P 32c5ad23e4204495a9f41f9a7dd668ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:08:06.275477 5506 raft_consensus.cc:482] T f4691e48b7ef4cbfb02901dfd12f84cb P 32c5ad23e4204495a9f41f9a7dd668ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:08:06.275589 5506 raft_consensus.cc:3037] T f4691e48b7ef4cbfb02901dfd12f84cb P 32c5ad23e4204495a9f41f9a7dd668ad [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:08:06.278870 5506 raft_consensus.cc:504] T f4691e48b7ef4cbfb02901dfd12f84cb P 32c5ad23e4204495a9f41f9a7dd668ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } }
I20260809 15:08:06.280742 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4691e48b7ef4cbfb02901dfd12f84cb" candidate_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5565ec2abbf846858fef0f8311f80171"
I20260809 15:08:06.280791 5140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4691e48b7ef4cbfb02901dfd12f84cb" candidate_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17f925643d44003bf98d29f94a48a4b"
I20260809 15:08:06.281311 5386 raft_consensus.cc:3037] T f4691e48b7ef4cbfb02901dfd12f84cb P 5565ec2abbf846858fef0f8311f80171 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:08:06.281479 5140 raft_consensus.cc:3037] T f4691e48b7ef4cbfb02901dfd12f84cb P e17f925643d44003bf98d29f94a48a4b [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:08:06.281602 5506 leader_election.cc:288] T f4691e48b7ef4cbfb02901dfd12f84cb P 32c5ad23e4204495a9f41f9a7dd668ad [CANDIDATE]: Term 1 election: Requested vote from peers e17f925643d44003bf98d29f94a48a4b (127.24.183.194:46433), 5565ec2abbf846858fef0f8311f80171 (127.24.183.195:36621)
I20260809 15:08:06.284458 5140 raft_consensus.cc:2454] T f4691e48b7ef4cbfb02901dfd12f84cb P e17f925643d44003bf98d29f94a48a4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32c5ad23e4204495a9f41f9a7dd668ad in term 1.
I20260809 15:08:06.284811 5386 raft_consensus.cc:2454] T f4691e48b7ef4cbfb02901dfd12f84cb P 5565ec2abbf846858fef0f8311f80171 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32c5ad23e4204495a9f41f9a7dd668ad in term 1.
I20260809 15:08:06.284907 4788 leader_election.cc:302] T f4691e48b7ef4cbfb02901dfd12f84cb P 32c5ad23e4204495a9f41f9a7dd668ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32c5ad23e4204495a9f41f9a7dd668ad, e17f925643d44003bf98d29f94a48a4b; no voters:
I20260809 15:08:06.285291 5506 raft_consensus.cc:2781] T f4691e48b7ef4cbfb02901dfd12f84cb P 32c5ad23e4204495a9f41f9a7dd668ad [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:08:06.285458 5506 raft_consensus.cc:686] T f4691e48b7ef4cbfb02901dfd12f84cb P 32c5ad23e4204495a9f41f9a7dd668ad [term 1 LEADER]: Becoming Leader. State: Replica: 32c5ad23e4204495a9f41f9a7dd668ad, State: Running, Role: LEADER
I20260809 15:08:06.285833 5506 consensus_queue.cc:237] T f4691e48b7ef4cbfb02901dfd12f84cb P 32c5ad23e4204495a9f41f9a7dd668ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } }
I20260809 15:08:06.287564 5512 tablet_replica.cc:439] T f4691e48b7ef4cbfb02901dfd12f84cb P 32c5ad23e4204495a9f41f9a7dd668ad: TxnStatusTablet state changed. Reason: New leader 32c5ad23e4204495a9f41f9a7dd668ad. Latest consensus state: current_term: 1 leader_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } } }
I20260809 15:08:06.287919 5512 tablet_replica.cc:442] T f4691e48b7ef4cbfb02901dfd12f84cb P 32c5ad23e4204495a9f41f9a7dd668ad: This TxnStatusTablet replica's current role is: LEADER
I20260809 15:08:06.289176 4625 catalog_manager.cc:5495] T f4691e48b7ef4cbfb02901dfd12f84cb P 32c5ad23e4204495a9f41f9a7dd668ad reported cstate change: term changed from 0 to 1, leader changed from <none> to 32c5ad23e4204495a9f41f9a7dd668ad (127.24.183.193). New cstate: current_term: 1 leader_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } health_report { overall_health: UNKNOWN } } }
I20260809 15:08:06.296284 5561 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 15:08:06.296572 5561 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 15:08:06.311406 5494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:06.304999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40780 (local address 127.24.183.193:39465)
0809 15:08:06.305191 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:06.305196 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:06.305274 (+ 78us) server_negotiation.cc:407] Connection header received
0809 15:08:06.305541 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:06.305544 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:06.305610 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:06.305692 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:08:06.306540 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:06.307450 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:06.309401 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:06.309928 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:06.310062 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:06.310615 (+ 553us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:08:06.310770 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:08:06.310860 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:06.311040 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 15:08:06.311114 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 15:08:06.311563 5554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:06.304931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:39465 (local address 127.0.0.1:40780)
0809 15:08:06.305180 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:06.305202 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:06.305365 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:06.305737 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:06.305742 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:06.305782 (+ 40us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 15:08:06.306286 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:06.306301 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:06.307600 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:06.307605 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:06.309216 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:06.309233 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:06.310107 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:06.310113 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:06.310331 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:06.310412 (+ 81us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 15:08:06.310883 (+ 471us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 15:08:06.310913 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 15:08:06.310997 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 15:08:06.311161 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 15:08:06.316454 5386 raft_consensus.cc:1261] T f4691e48b7ef4cbfb02901dfd12f84cb P 5565ec2abbf846858fef0f8311f80171 [term 1 FOLLOWER]: Refusing update from remote peer 32c5ad23e4204495a9f41f9a7dd668ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:08:06.316565 5140 raft_consensus.cc:1261] T f4691e48b7ef4cbfb02901dfd12f84cb P e17f925643d44003bf98d29f94a48a4b [term 1 FOLLOWER]: Refusing update from remote peer 32c5ad23e4204495a9f41f9a7dd668ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:08:06.317167 5512 consensus_queue.cc:1035] T f4691e48b7ef4cbfb02901dfd12f84cb P 32c5ad23e4204495a9f41f9a7dd668ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:08:06.317497 5506 consensus_queue.cc:1035] T f4691e48b7ef4cbfb02901dfd12f84cb P 32c5ad23e4204495a9f41f9a7dd668ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:08:06.320524 5508 tablet_replica.cc:439] T f4691e48b7ef4cbfb02901dfd12f84cb P e17f925643d44003bf98d29f94a48a4b: TxnStatusTablet state changed. Reason: New leader 32c5ad23e4204495a9f41f9a7dd668ad. Latest consensus state: current_term: 1 leader_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } } }
I20260809 15:08:06.320962 5508 tablet_replica.cc:442] T f4691e48b7ef4cbfb02901dfd12f84cb P e17f925643d44003bf98d29f94a48a4b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:08:06.324071 5526 tablet_replica.cc:439] T f4691e48b7ef4cbfb02901dfd12f84cb P 32c5ad23e4204495a9f41f9a7dd668ad: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } } }
I20260809 15:08:06.324718 5526 tablet_replica.cc:442] T f4691e48b7ef4cbfb02901dfd12f84cb P 32c5ad23e4204495a9f41f9a7dd668ad: This TxnStatusTablet replica's current role is: LEADER
I20260809 15:08:06.328446 5503 tablet_replica.cc:439] T f4691e48b7ef4cbfb02901dfd12f84cb P 5565ec2abbf846858fef0f8311f80171: TxnStatusTablet state changed. Reason: New leader 32c5ad23e4204495a9f41f9a7dd668ad. Latest consensus state: current_term: 1 leader_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } } }
I20260809 15:08:06.328776 5503 tablet_replica.cc:442] T f4691e48b7ef4cbfb02901dfd12f84cb P 5565ec2abbf846858fef0f8311f80171: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:08:06.329391 5526 tablet_replica.cc:439] T f4691e48b7ef4cbfb02901dfd12f84cb P 32c5ad23e4204495a9f41f9a7dd668ad: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } } }
I20260809 15:08:06.329672 5526 tablet_replica.cc:442] T f4691e48b7ef4cbfb02901dfd12f84cb P 32c5ad23e4204495a9f41f9a7dd668ad: This TxnStatusTablet replica's current role is: LEADER
I20260809 15:08:06.330444 5508 tablet_replica.cc:439] T f4691e48b7ef4cbfb02901dfd12f84cb P e17f925643d44003bf98d29f94a48a4b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } } }
I20260809 15:08:06.330742 5508 tablet_replica.cc:442] T f4691e48b7ef4cbfb02901dfd12f84cb P e17f925643d44003bf98d29f94a48a4b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:08:06.332742 5503 tablet_replica.cc:439] T f4691e48b7ef4cbfb02901dfd12f84cb P 5565ec2abbf846858fef0f8311f80171: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } } }
I20260809 15:08:06.333003 5503 tablet_replica.cc:442] T f4691e48b7ef4cbfb02901dfd12f84cb P 5565ec2abbf846858fef0f8311f80171: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:08:06.356882 5494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:06.351019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:45189 (local address 127.24.183.193:39465)
0809 15:08:06.351170 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:06.351174 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:06.351915 (+ 741us) server_negotiation.cc:407] Connection header received
0809 15:08:06.352098 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:06.352101 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:06.352162 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:06.352236 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:08:06.352757 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:06.353540 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:06.355007 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:06.355466 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:06.355585 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:06.356072 (+ 487us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:08:06.356234 (+ 162us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:08:06.356314 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:06.356459 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 15:08:06.356534 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 15:08:06.357010 5572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:06.351147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:39465 (local address 127.24.183.193:45189)
0809 15:08:06.351821 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:06.351838 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:06.351986 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:06.352263 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:06.352267 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:06.352277 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 15:08:06.352612 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:06.352622 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:06.353662 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:06.353668 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:06.354869 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:06.354882 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:06.355651 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:06.355658 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:06.355842 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:06.355910 (+ 68us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 15:08:06.356337 (+ 427us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 15:08:06.356348 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 15:08:06.356426 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 15:08:06.356582 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":348,"threads_started":1}
I20260809 15:08:06.370011 5140 raft_consensus.cc:1261] T 798f421b267541b299e899058ebafba1 P e17f925643d44003bf98d29f94a48a4b [term 1 FOLLOWER]: Refusing update from remote peer 32c5ad23e4204495a9f41f9a7dd668ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:08:06.370628 5386 raft_consensus.cc:1261] T 798f421b267541b299e899058ebafba1 P 5565ec2abbf846858fef0f8311f80171 [term 1 FOLLOWER]: Refusing update from remote peer 32c5ad23e4204495a9f41f9a7dd668ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:08:06.370599 5506 consensus_queue.cc:1035] T 798f421b267541b299e899058ebafba1 P 32c5ad23e4204495a9f41f9a7dd668ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:08:06.371232 5506 consensus_queue.cc:1035] T 798f421b267541b299e899058ebafba1 P 32c5ad23e4204495a9f41f9a7dd668ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:08:06.472445 4627 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:08:06.472770 4627 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:08:06.476938 4627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 36c2e1a7410f4f569d153b1f1b6cc236: Sending DeleteTablet for 3 replicas of tablet 798f421b267541b299e899058ebafba1
I20260809 15:08:06.478261 5098 tablet_service.cc:1514] Processing DeleteTablet for tablet 798f421b267541b299e899058ebafba1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:08:06 UTC) from {username='oryx', principal='oryx/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:52834
I20260809 15:08:06.478615 5532 tablet_replica.cc:330] T 798f421b267541b299e899058ebafba1 P e17f925643d44003bf98d29f94a48a4b: stopping tablet replica
I20260809 15:08:06.478854 5532 raft_consensus.cc:2229] T 798f421b267541b299e899058ebafba1 P e17f925643d44003bf98d29f94a48a4b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:08:06.479038 5332 tablet_service.cc:1514] Processing DeleteTablet for tablet 798f421b267541b299e899058ebafba1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:08:06 UTC) from {username='oryx', principal='oryx/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:60588
I20260809 15:08:06.479157 5532 raft_consensus.cc:2258] T 798f421b267541b299e899058ebafba1 P e17f925643d44003bf98d29f94a48a4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:08:06.479351 5533 tablet_replica.cc:330] T 798f421b267541b299e899058ebafba1 P 5565ec2abbf846858fef0f8311f80171: stopping tablet replica
I20260809 15:08:06.479558 5533 raft_consensus.cc:2229] T 798f421b267541b299e899058ebafba1 P 5565ec2abbf846858fef0f8311f80171 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:08:06.479792 5533 raft_consensus.cc:2258] T 798f421b267541b299e899058ebafba1 P 5565ec2abbf846858fef0f8311f80171 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:08:06.478276 4869 tablet_service.cc:1514] Processing DeleteTablet for tablet 798f421b267541b299e899058ebafba1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:08:06 UTC) from {username='oryx', principal='oryx/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:40770
I20260809 15:08:06.481223 5535 tablet_replica.cc:330] T 798f421b267541b299e899058ebafba1 P 32c5ad23e4204495a9f41f9a7dd668ad: stopping tablet replica
I20260809 15:08:06.481482 5532 ts_tablet_manager.cc:1905] T 798f421b267541b299e899058ebafba1 P e17f925643d44003bf98d29f94a48a4b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:08:06.481673 5535 raft_consensus.cc:2229] T 798f421b267541b299e899058ebafba1 P 32c5ad23e4204495a9f41f9a7dd668ad [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:08:06.482409 5535 raft_consensus.cc:2258] T 798f421b267541b299e899058ebafba1 P 32c5ad23e4204495a9f41f9a7dd668ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:08:06.482741 5533 ts_tablet_manager.cc:1905] T 798f421b267541b299e899058ebafba1 P 5565ec2abbf846858fef0f8311f80171: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:08:06.489226 5535 ts_tablet_manager.cc:1905] T 798f421b267541b299e899058ebafba1 P 32c5ad23e4204495a9f41f9a7dd668ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:08:06.490360 5532 ts_tablet_manager.cc:1918] T 798f421b267541b299e899058ebafba1 P e17f925643d44003bf98d29f94a48a4b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:08:06.490644 5532 log.cc:1196] T 798f421b267541b299e899058ebafba1 P e17f925643d44003bf98d29f94a48a4b: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-1/wal/wals/798f421b267541b299e899058ebafba1
I20260809 15:08:06.491199 5532 ts_tablet_manager.cc:1939] T 798f421b267541b299e899058ebafba1 P e17f925643d44003bf98d29f94a48a4b: Deleting consensus metadata
I20260809 15:08:06.498216 5535 ts_tablet_manager.cc:1918] T 798f421b267541b299e899058ebafba1 P 32c5ad23e4204495a9f41f9a7dd668ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:08:06.498423 5535 log.cc:1196] T 798f421b267541b299e899058ebafba1 P 32c5ad23e4204495a9f41f9a7dd668ad: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-0/wal/wals/798f421b267541b299e899058ebafba1
I20260809 15:08:06.498903 5535 ts_tablet_manager.cc:1939] T 798f421b267541b299e899058ebafba1 P 32c5ad23e4204495a9f41f9a7dd668ad: Deleting consensus metadata
I20260809 15:08:06.501022 4603 catalog_manager.cc:4841] TS e17f925643d44003bf98d29f94a48a4b (127.24.183.194:46433): tablet 798f421b267541b299e899058ebafba1 (table test-table [id=5eade2520f7e4443a267e32608f8b9f8]) successfully deleted
I20260809 15:08:06.503595 5533 ts_tablet_manager.cc:1918] T 798f421b267541b299e899058ebafba1 P 5565ec2abbf846858fef0f8311f80171: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:08:06.503844 5533 log.cc:1196] T 798f421b267541b299e899058ebafba1 P 5565ec2abbf846858fef0f8311f80171: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-2/wal/wals/798f421b267541b299e899058ebafba1
I20260809 15:08:06.504279 5533 ts_tablet_manager.cc:1939] T 798f421b267541b299e899058ebafba1 P 5565ec2abbf846858fef0f8311f80171: Deleting consensus metadata
I20260809 15:08:06.505450 4605 catalog_manager.cc:4841] TS 5565ec2abbf846858fef0f8311f80171 (127.24.183.195:36621): tablet 798f421b267541b299e899058ebafba1 (table test-table [id=5eade2520f7e4443a267e32608f8b9f8]) successfully deleted
I20260809 15:08:06.508338 4605 catalog_manager.cc:4841] TS 32c5ad23e4204495a9f41f9a7dd668ad (127.24.183.193:39465): tablet 798f421b267541b299e899058ebafba1 (table test-table [id=5eade2520f7e4443a267e32608f8b9f8]) successfully deleted
Aug 09 15:08:06 dist-test-slave-nj21 krb5kdc[4555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288086, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 15:08:06 dist-test-slave-nj21 krb5kdc[4555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:06 dist-test-slave-nj21 krb5kdc[4555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 15:08:06.603049 5553 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:06.587958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35812 (local address 127.24.183.254:34295)
0809 15:08:06.588141 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:06.588147 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:06.590403 (+ 2256us) server_negotiation.cc:407] Connection header received
0809 15:08:06.590695 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:06.590739 (+ 44us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:06.590829 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:06.590925 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:06.592902 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:06.594027 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:06.595972 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:06.596665 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:06.596758 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:06.602760 (+ 6002us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35812: BlockingRecv error: recv got EOF from 127.0.0.1:35812 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 15:08:06 dist-test-slave-nj21 krb5kdc[4555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288086, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM
I20260809 15:08:06.623368 5553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:06.607248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35828 (local address 127.24.183.254:34295)
0809 15:08:06.607463 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:06.607470 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:06.607710 (+ 240us) server_negotiation.cc:407] Connection header received
0809 15:08:06.607910 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:06.607915 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:06.607986 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:06.608069 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:06.609453 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:06.610537 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:06.611762 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:06.612191 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:06.612308 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:06.617896 (+ 5588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:06.617901 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:06.617905 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:08:06.617961 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:06.620689 (+ 2728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:06.621466 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:06.621470 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:06.621472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:06.621674 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:06.622060 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:06.622063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:06.622065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:06.622340 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:06.622582 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:06.622870 (+ 288us) server_negotiation.cc:299] Negotiation successful
0809 15:08:06.623071 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 15:08:06.628334 4627 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35828:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:08:06.629375 4627 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:08:06.636600 4869 tablet_service.cc:1467] Processing CreateTablet for tablet 912d70dbb373462682c5840e6305890b (DEFAULT_TABLE table=test-table [id=627ce0ee5e2d4d0583fa5675ed283bd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:08:06.637041 5332 tablet_service.cc:1467] Processing CreateTablet for tablet 912d70dbb373462682c5840e6305890b (DEFAULT_TABLE table=test-table [id=627ce0ee5e2d4d0583fa5675ed283bd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:08:06.637274 4869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 912d70dbb373462682c5840e6305890b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:08:06.637511 5332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 912d70dbb373462682c5840e6305890b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:08:06.640417 5098 tablet_service.cc:1467] Processing CreateTablet for tablet 912d70dbb373462682c5840e6305890b (DEFAULT_TABLE table=test-table [id=627ce0ee5e2d4d0583fa5675ed283bd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:08:06.640892 5098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 912d70dbb373462682c5840e6305890b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:08:06.644690 5495 tablet_bootstrap.cc:492] T 912d70dbb373462682c5840e6305890b P 5565ec2abbf846858fef0f8311f80171: Bootstrap starting.
I20260809 15:08:06.645375 5496 tablet_bootstrap.cc:492] T 912d70dbb373462682c5840e6305890b P 32c5ad23e4204495a9f41f9a7dd668ad: Bootstrap starting.
I20260809 15:08:06.646631 5495 tablet_bootstrap.cc:654] T 912d70dbb373462682c5840e6305890b P 5565ec2abbf846858fef0f8311f80171: Neither blocks nor log segments found. Creating new log.
I20260809 15:08:06.647532 5496 tablet_bootstrap.cc:654] T 912d70dbb373462682c5840e6305890b P 32c5ad23e4204495a9f41f9a7dd668ad: Neither blocks nor log segments found. Creating new log.
I20260809 15:08:06.649374 5497 tablet_bootstrap.cc:492] T 912d70dbb373462682c5840e6305890b P e17f925643d44003bf98d29f94a48a4b: Bootstrap starting.
I20260809 15:08:06.652352 5497 tablet_bootstrap.cc:654] T 912d70dbb373462682c5840e6305890b P e17f925643d44003bf98d29f94a48a4b: Neither blocks nor log segments found. Creating new log.
I20260809 15:08:06.656474 5497 tablet_bootstrap.cc:492] T 912d70dbb373462682c5840e6305890b P e17f925643d44003bf98d29f94a48a4b: No bootstrap required, opened a new log
I20260809 15:08:06.656638 5497 ts_tablet_manager.cc:1397] T 912d70dbb373462682c5840e6305890b P e17f925643d44003bf98d29f94a48a4b: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260809 15:08:06.658529 5497 raft_consensus.cc:348] T 912d70dbb373462682c5840e6305890b P e17f925643d44003bf98d29f94a48a4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } }
I20260809 15:08:06.658897 5497 raft_consensus.cc:374] T 912d70dbb373462682c5840e6305890b P e17f925643d44003bf98d29f94a48a4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:08:06.658998 5497 raft_consensus.cc:729] T 912d70dbb373462682c5840e6305890b P e17f925643d44003bf98d29f94a48a4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e17f925643d44003bf98d29f94a48a4b, State: Initialized, Role: FOLLOWER
I20260809 15:08:06.659276 5497 consensus_queue.cc:260] T 912d70dbb373462682c5840e6305890b P e17f925643d44003bf98d29f94a48a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } }
I20260809 15:08:06.659909 5497 ts_tablet_manager.cc:1428] T 912d70dbb373462682c5840e6305890b P e17f925643d44003bf98d29f94a48a4b: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260809 15:08:06.660653 5496 tablet_bootstrap.cc:492] T 912d70dbb373462682c5840e6305890b P 32c5ad23e4204495a9f41f9a7dd668ad: No bootstrap required, opened a new log
I20260809 15:08:06.660833 5496 ts_tablet_manager.cc:1397] T 912d70dbb373462682c5840e6305890b P 32c5ad23e4204495a9f41f9a7dd668ad: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.005s
I20260809 15:08:06.662655 5495 tablet_bootstrap.cc:492] T 912d70dbb373462682c5840e6305890b P 5565ec2abbf846858fef0f8311f80171: No bootstrap required, opened a new log
I20260809 15:08:06.662890 5495 ts_tablet_manager.cc:1397] T 912d70dbb373462682c5840e6305890b P 5565ec2abbf846858fef0f8311f80171: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.012s
I20260809 15:08:06.661767 5496 raft_consensus.cc:348] T 912d70dbb373462682c5840e6305890b P 32c5ad23e4204495a9f41f9a7dd668ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } }
I20260809 15:08:06.663776 5495 raft_consensus.cc:348] T 912d70dbb373462682c5840e6305890b P 5565ec2abbf846858fef0f8311f80171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } }
I20260809 15:08:06.664167 5495 raft_consensus.cc:374] T 912d70dbb373462682c5840e6305890b P 5565ec2abbf846858fef0f8311f80171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:08:06.664040 5496 raft_consensus.cc:374] T 912d70dbb373462682c5840e6305890b P 32c5ad23e4204495a9f41f9a7dd668ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:08:06.664281 5495 raft_consensus.cc:729] T 912d70dbb373462682c5840e6305890b P 5565ec2abbf846858fef0f8311f80171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5565ec2abbf846858fef0f8311f80171, State: Initialized, Role: FOLLOWER
I20260809 15:08:06.664299 5496 raft_consensus.cc:729] T 912d70dbb373462682c5840e6305890b P 32c5ad23e4204495a9f41f9a7dd668ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32c5ad23e4204495a9f41f9a7dd668ad, State: Initialized, Role: FOLLOWER
I20260809 15:08:06.664568 5496 consensus_queue.cc:260] T 912d70dbb373462682c5840e6305890b P 32c5ad23e4204495a9f41f9a7dd668ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } }
I20260809 15:08:06.664517 5495 consensus_queue.cc:260] T 912d70dbb373462682c5840e6305890b P 5565ec2abbf846858fef0f8311f80171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } }
I20260809 15:08:06.665206 5496 ts_tablet_manager.cc:1428] T 912d70dbb373462682c5840e6305890b P 32c5ad23e4204495a9f41f9a7dd668ad: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260809 15:08:06.666217 5495 ts_tablet_manager.cc:1428] T 912d70dbb373462682c5840e6305890b P 5565ec2abbf846858fef0f8311f80171: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 15:08:06.824067 5512 raft_consensus.cc:482] T 912d70dbb373462682c5840e6305890b P 32c5ad23e4204495a9f41f9a7dd668ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:08:06.824287 5512 raft_consensus.cc:504] T 912d70dbb373462682c5840e6305890b P 32c5ad23e4204495a9f41f9a7dd668ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } }
I20260809 15:08:06.825029 5512 leader_election.cc:288] T 912d70dbb373462682c5840e6305890b P 32c5ad23e4204495a9f41f9a7dd668ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e17f925643d44003bf98d29f94a48a4b (127.24.183.194:46433), 5565ec2abbf846858fef0f8311f80171 (127.24.183.195:36621)
I20260809 15:08:06.825665 5139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "912d70dbb373462682c5840e6305890b" candidate_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17f925643d44003bf98d29f94a48a4b" is_pre_election: true
I20260809 15:08:06.825700 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "912d70dbb373462682c5840e6305890b" candidate_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5565ec2abbf846858fef0f8311f80171" is_pre_election: true
I20260809 15:08:06.825954 5139 raft_consensus.cc:2454] T 912d70dbb373462682c5840e6305890b P e17f925643d44003bf98d29f94a48a4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32c5ad23e4204495a9f41f9a7dd668ad in term 0.
I20260809 15:08:06.825961 5386 raft_consensus.cc:2454] T 912d70dbb373462682c5840e6305890b P 5565ec2abbf846858fef0f8311f80171 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32c5ad23e4204495a9f41f9a7dd668ad in term 0.
I20260809 15:08:06.826418 4788 leader_election.cc:302] T 912d70dbb373462682c5840e6305890b P 32c5ad23e4204495a9f41f9a7dd668ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32c5ad23e4204495a9f41f9a7dd668ad, e17f925643d44003bf98d29f94a48a4b; no voters:
I20260809 15:08:06.826713 5512 raft_consensus.cc:2781] T 912d70dbb373462682c5840e6305890b P 32c5ad23e4204495a9f41f9a7dd668ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:08:06.826853 5512 raft_consensus.cc:482] T 912d70dbb373462682c5840e6305890b P 32c5ad23e4204495a9f41f9a7dd668ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:08:06.826953 5512 raft_consensus.cc:3037] T 912d70dbb373462682c5840e6305890b P 32c5ad23e4204495a9f41f9a7dd668ad [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:08:06.830122 5512 raft_consensus.cc:504] T 912d70dbb373462682c5840e6305890b P 32c5ad23e4204495a9f41f9a7dd668ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } }
I20260809 15:08:06.830749 5512 leader_election.cc:288] T 912d70dbb373462682c5840e6305890b P 32c5ad23e4204495a9f41f9a7dd668ad [CANDIDATE]: Term 1 election: Requested vote from peers e17f925643d44003bf98d29f94a48a4b (127.24.183.194:46433), 5565ec2abbf846858fef0f8311f80171 (127.24.183.195:36621)
I20260809 15:08:06.831302 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "912d70dbb373462682c5840e6305890b" candidate_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5565ec2abbf846858fef0f8311f80171"
I20260809 15:08:06.831372 5139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "912d70dbb373462682c5840e6305890b" candidate_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17f925643d44003bf98d29f94a48a4b"
I20260809 15:08:06.831508 5386 raft_consensus.cc:3037] T 912d70dbb373462682c5840e6305890b P 5565ec2abbf846858fef0f8311f80171 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:08:06.831637 5139 raft_consensus.cc:3037] T 912d70dbb373462682c5840e6305890b P e17f925643d44003bf98d29f94a48a4b [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:08:06.833730 5139 raft_consensus.cc:2454] T 912d70dbb373462682c5840e6305890b P e17f925643d44003bf98d29f94a48a4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32c5ad23e4204495a9f41f9a7dd668ad in term 1.
I20260809 15:08:06.834151 4788 leader_election.cc:302] T 912d70dbb373462682c5840e6305890b P 32c5ad23e4204495a9f41f9a7dd668ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32c5ad23e4204495a9f41f9a7dd668ad, e17f925643d44003bf98d29f94a48a4b; no voters:
I20260809 15:08:06.834352 5386 raft_consensus.cc:2454] T 912d70dbb373462682c5840e6305890b P 5565ec2abbf846858fef0f8311f80171 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32c5ad23e4204495a9f41f9a7dd668ad in term 1.
I20260809 15:08:06.834438 5512 raft_consensus.cc:2781] T 912d70dbb373462682c5840e6305890b P 32c5ad23e4204495a9f41f9a7dd668ad [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:08:06.834604 5512 raft_consensus.cc:686] T 912d70dbb373462682c5840e6305890b P 32c5ad23e4204495a9f41f9a7dd668ad [term 1 LEADER]: Becoming Leader. State: Replica: 32c5ad23e4204495a9f41f9a7dd668ad, State: Running, Role: LEADER
I20260809 15:08:06.834944 5512 consensus_queue.cc:237] T 912d70dbb373462682c5840e6305890b P 32c5ad23e4204495a9f41f9a7dd668ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } }
I20260809 15:08:06.837867 4628 catalog_manager.cc:5495] T 912d70dbb373462682c5840e6305890b P 32c5ad23e4204495a9f41f9a7dd668ad reported cstate change: term changed from 0 to 1, leader changed from <none> to 32c5ad23e4204495a9f41f9a7dd668ad (127.24.183.193). New cstate: current_term: 1 leader_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } health_report { overall_health: UNKNOWN } } }
I20260809 15:08:06.854879 5494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:06.848944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40792 (local address 127.24.183.193:39465)
0809 15:08:06.849164 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:06.849170 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:06.849618 (+ 448us) server_negotiation.cc:407] Connection header received
0809 15:08:06.849686 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:06.849689 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:06.849759 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:06.849832 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:08:06.850241 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:06.851137 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:06.853083 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:06.853580 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:06.853683 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:06.854206 (+ 523us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:08:06.854321 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:08:06.854386 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:06.854522 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 15:08:06.854597 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 15:08:06.857690 5139 raft_consensus.cc:1261] T 912d70dbb373462682c5840e6305890b P e17f925643d44003bf98d29f94a48a4b [term 1 FOLLOWER]: Refusing update from remote peer 32c5ad23e4204495a9f41f9a7dd668ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:08:06.857905 5386 raft_consensus.cc:1261] T 912d70dbb373462682c5840e6305890b P 5565ec2abbf846858fef0f8311f80171 [term 1 FOLLOWER]: Refusing update from remote peer 32c5ad23e4204495a9f41f9a7dd668ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:08:06.858232 5512 consensus_queue.cc:1035] T 912d70dbb373462682c5840e6305890b P 32c5ad23e4204495a9f41f9a7dd668ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:08:06.858495 5506 consensus_queue.cc:1035] T 912d70dbb373462682c5840e6305890b P 32c5ad23e4204495a9f41f9a7dd668ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:08:06.864142 5519 mvcc.cc:205] Tried to move back new op lower bound from 7316636003763130368 to 7316636003678486528. Current Snapshot: MvccSnapshot[applied={T|T < 7316636003763130368}]
I20260809 15:08:06.870863 4625 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:08:06.871093 4625 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:08:06.873991 4625 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 36c2e1a7410f4f569d153b1f1b6cc236: Sending DeleteTablet for 3 replicas of tablet 912d70dbb373462682c5840e6305890b
I20260809 15:08:06.874703 4869 tablet_service.cc:1514] Processing DeleteTablet for tablet 912d70dbb373462682c5840e6305890b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:08:06 UTC) from {username='oryx', principal='oryx/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:40770
I20260809 15:08:06.874717 5098 tablet_service.cc:1514] Processing DeleteTablet for tablet 912d70dbb373462682c5840e6305890b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:08:06 UTC) from {username='oryx', principal='oryx/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:52834
I20260809 15:08:06.875003 5532 tablet_replica.cc:330] T 912d70dbb373462682c5840e6305890b P e17f925643d44003bf98d29f94a48a4b: stopping tablet replica
I20260809 15:08:06.875016 5535 tablet_replica.cc:330] T 912d70dbb373462682c5840e6305890b P 32c5ad23e4204495a9f41f9a7dd668ad: stopping tablet replica
I20260809 15:08:06.875177 5532 raft_consensus.cc:2229] T 912d70dbb373462682c5840e6305890b P e17f925643d44003bf98d29f94a48a4b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:08:06.875202 5535 raft_consensus.cc:2229] T 912d70dbb373462682c5840e6305890b P 32c5ad23e4204495a9f41f9a7dd668ad [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:08:06.875401 5532 raft_consensus.cc:2258] T 912d70dbb373462682c5840e6305890b P e17f925643d44003bf98d29f94a48a4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:08:06.875547 5535 raft_consensus.cc:2258] T 912d70dbb373462682c5840e6305890b P 32c5ad23e4204495a9f41f9a7dd668ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:08:06.876550 5332 tablet_service.cc:1514] Processing DeleteTablet for tablet 912d70dbb373462682c5840e6305890b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:08:06 UTC) from {username='oryx', principal='oryx/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:60588
I20260809 15:08:06.876858 5533 tablet_replica.cc:330] T 912d70dbb373462682c5840e6305890b P 5565ec2abbf846858fef0f8311f80171: stopping tablet replica
I20260809 15:08:06.877050 5533 raft_consensus.cc:2229] T 912d70dbb373462682c5840e6305890b P 5565ec2abbf846858fef0f8311f80171 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:08:06.877298 5533 raft_consensus.cc:2258] T 912d70dbb373462682c5840e6305890b P 5565ec2abbf846858fef0f8311f80171 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:08:06.877611 4625 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35828:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:08:06.878330 4625 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:08:06.878826 5533 ts_tablet_manager.cc:1905] T 912d70dbb373462682c5840e6305890b P 5565ec2abbf846858fef0f8311f80171: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:08:06.880563 5535 ts_tablet_manager.cc:1905] T 912d70dbb373462682c5840e6305890b P 32c5ad23e4204495a9f41f9a7dd668ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:08:06.885589 5533 ts_tablet_manager.cc:1918] T 912d70dbb373462682c5840e6305890b P 5565ec2abbf846858fef0f8311f80171: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:08:06.886169 5098 tablet_service.cc:1467] Processing CreateTablet for tablet 12baa1a998374a229980bf6a1d02a950 (DEFAULT_TABLE table=test-table [id=73a4b295c70a408881caa19588cae545]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:08:06.886628 5098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12baa1a998374a229980bf6a1d02a950. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:08:06.886605 4869 tablet_service.cc:1467] Processing CreateTablet for tablet 12baa1a998374a229980bf6a1d02a950 (DEFAULT_TABLE table=test-table [id=73a4b295c70a408881caa19588cae545]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:08:06.887003 4869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12baa1a998374a229980bf6a1d02a950. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:08:06.887173 5535 ts_tablet_manager.cc:1918] T 912d70dbb373462682c5840e6305890b P 32c5ad23e4204495a9f41f9a7dd668ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:08:06.887300 5535 log.cc:1196] T 912d70dbb373462682c5840e6305890b P 32c5ad23e4204495a9f41f9a7dd668ad: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-0/wal/wals/912d70dbb373462682c5840e6305890b
I20260809 15:08:06.887710 5535 ts_tablet_manager.cc:1939] T 912d70dbb373462682c5840e6305890b P 32c5ad23e4204495a9f41f9a7dd668ad: Deleting consensus metadata
I20260809 15:08:06.890736 4605 catalog_manager.cc:4841] TS 32c5ad23e4204495a9f41f9a7dd668ad (127.24.183.193:39465): tablet 912d70dbb373462682c5840e6305890b (table test-table [id=627ce0ee5e2d4d0583fa5675ed283bd4]) successfully deleted
I20260809 15:08:06.894762 5496 tablet_bootstrap.cc:492] T 12baa1a998374a229980bf6a1d02a950 P 32c5ad23e4204495a9f41f9a7dd668ad: Bootstrap starting.
I20260809 15:08:06.895593 5497 tablet_bootstrap.cc:492] T 12baa1a998374a229980bf6a1d02a950 P e17f925643d44003bf98d29f94a48a4b: Bootstrap starting.
I20260809 15:08:06.896617 5496 tablet_bootstrap.cc:654] T 12baa1a998374a229980bf6a1d02a950 P 32c5ad23e4204495a9f41f9a7dd668ad: Neither blocks nor log segments found. Creating new log.
I20260809 15:08:06.897267 5497 tablet_bootstrap.cc:654] T 12baa1a998374a229980bf6a1d02a950 P e17f925643d44003bf98d29f94a48a4b: Neither blocks nor log segments found. Creating new log.
I20260809 15:08:06.890094 5332 tablet_service.cc:1467] Processing CreateTablet for tablet 12baa1a998374a229980bf6a1d02a950 (DEFAULT_TABLE table=test-table [id=73a4b295c70a408881caa19588cae545]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:08:06.900686 5332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12baa1a998374a229980bf6a1d02a950. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:08:06.904336 5533 log.cc:1196] T 912d70dbb373462682c5840e6305890b P 5565ec2abbf846858fef0f8311f80171: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-2/wal/wals/912d70dbb373462682c5840e6305890b
I20260809 15:08:06.904973 5532 ts_tablet_manager.cc:1905] T 912d70dbb373462682c5840e6305890b P e17f925643d44003bf98d29f94a48a4b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:08:06.905592 5533 ts_tablet_manager.cc:1939] T 912d70dbb373462682c5840e6305890b P 5565ec2abbf846858fef0f8311f80171: Deleting consensus metadata
I20260809 15:08:06.907127 4605 catalog_manager.cc:4841] TS 5565ec2abbf846858fef0f8311f80171 (127.24.183.195:36621): tablet 912d70dbb373462682c5840e6305890b (table test-table [id=627ce0ee5e2d4d0583fa5675ed283bd4]) successfully deleted
I20260809 15:08:06.908595 5495 tablet_bootstrap.cc:492] T 12baa1a998374a229980bf6a1d02a950 P 5565ec2abbf846858fef0f8311f80171: Bootstrap starting.
I20260809 15:08:06.910260 5495 tablet_bootstrap.cc:654] T 12baa1a998374a229980bf6a1d02a950 P 5565ec2abbf846858fef0f8311f80171: Neither blocks nor log segments found. Creating new log.
I20260809 15:08:06.911531 5532 ts_tablet_manager.cc:1918] T 912d70dbb373462682c5840e6305890b P e17f925643d44003bf98d29f94a48a4b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:08:06.912298 5496 tablet_bootstrap.cc:492] T 12baa1a998374a229980bf6a1d02a950 P 32c5ad23e4204495a9f41f9a7dd668ad: No bootstrap required, opened a new log
I20260809 15:08:06.912539 5496 ts_tablet_manager.cc:1397] T 12baa1a998374a229980bf6a1d02a950 P 32c5ad23e4204495a9f41f9a7dd668ad: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.005s
I20260809 15:08:06.912597 5532 log.cc:1196] T 912d70dbb373462682c5840e6305890b P e17f925643d44003bf98d29f94a48a4b: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-1/wal/wals/912d70dbb373462682c5840e6305890b
I20260809 15:08:06.913609 5532 ts_tablet_manager.cc:1939] T 912d70dbb373462682c5840e6305890b P e17f925643d44003bf98d29f94a48a4b: Deleting consensus metadata
I20260809 15:08:06.913569 5496 raft_consensus.cc:348] T 12baa1a998374a229980bf6a1d02a950 P 32c5ad23e4204495a9f41f9a7dd668ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } }
I20260809 15:08:06.914022 5496 raft_consensus.cc:374] T 12baa1a998374a229980bf6a1d02a950 P 32c5ad23e4204495a9f41f9a7dd668ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:08:06.914163 5496 raft_consensus.cc:729] T 12baa1a998374a229980bf6a1d02a950 P 32c5ad23e4204495a9f41f9a7dd668ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32c5ad23e4204495a9f41f9a7dd668ad, State: Initialized, Role: FOLLOWER
I20260809 15:08:06.914489 5496 consensus_queue.cc:260] T 12baa1a998374a229980bf6a1d02a950 P 32c5ad23e4204495a9f41f9a7dd668ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } }
I20260809 15:08:06.914875 4603 catalog_manager.cc:4841] TS e17f925643d44003bf98d29f94a48a4b (127.24.183.194:46433): tablet 912d70dbb373462682c5840e6305890b (table test-table [id=627ce0ee5e2d4d0583fa5675ed283bd4]) successfully deleted
I20260809 15:08:06.916256 5496 ts_tablet_manager.cc:1428] T 12baa1a998374a229980bf6a1d02a950 P 32c5ad23e4204495a9f41f9a7dd668ad: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260809 15:08:06.920935 5497 tablet_bootstrap.cc:492] T 12baa1a998374a229980bf6a1d02a950 P e17f925643d44003bf98d29f94a48a4b: No bootstrap required, opened a new log
I20260809 15:08:06.921001 5495 tablet_bootstrap.cc:492] T 12baa1a998374a229980bf6a1d02a950 P 5565ec2abbf846858fef0f8311f80171: No bootstrap required, opened a new log
I20260809 15:08:06.921190 5495 ts_tablet_manager.cc:1397] T 12baa1a998374a229980bf6a1d02a950 P 5565ec2abbf846858fef0f8311f80171: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.004s
I20260809 15:08:06.921200 5497 ts_tablet_manager.cc:1397] T 12baa1a998374a229980bf6a1d02a950 P e17f925643d44003bf98d29f94a48a4b: Time spent bootstrapping tablet: real 0.026s user 0.004s sys 0.006s
I20260809 15:08:06.922003 5495 raft_consensus.cc:348] T 12baa1a998374a229980bf6a1d02a950 P 5565ec2abbf846858fef0f8311f80171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } }
I20260809 15:08:06.922247 5495 raft_consensus.cc:374] T 12baa1a998374a229980bf6a1d02a950 P 5565ec2abbf846858fef0f8311f80171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:08:06.922137 5497 raft_consensus.cc:348] T 12baa1a998374a229980bf6a1d02a950 P e17f925643d44003bf98d29f94a48a4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } }
I20260809 15:08:06.922326 5495 raft_consensus.cc:729] T 12baa1a998374a229980bf6a1d02a950 P 5565ec2abbf846858fef0f8311f80171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5565ec2abbf846858fef0f8311f80171, State: Initialized, Role: FOLLOWER
I20260809 15:08:06.922385 5497 raft_consensus.cc:374] T 12baa1a998374a229980bf6a1d02a950 P e17f925643d44003bf98d29f94a48a4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:08:06.922473 5497 raft_consensus.cc:729] T 12baa1a998374a229980bf6a1d02a950 P e17f925643d44003bf98d29f94a48a4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e17f925643d44003bf98d29f94a48a4b, State: Initialized, Role: FOLLOWER
I20260809 15:08:06.922566 5495 consensus_queue.cc:260] T 12baa1a998374a229980bf6a1d02a950 P 5565ec2abbf846858fef0f8311f80171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } }
I20260809 15:08:06.922734 5497 consensus_queue.cc:260] T 12baa1a998374a229980bf6a1d02a950 P e17f925643d44003bf98d29f94a48a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } }
I20260809 15:08:06.923280 5497 ts_tablet_manager.cc:1428] T 12baa1a998374a229980bf6a1d02a950 P e17f925643d44003bf98d29f94a48a4b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 15:08:06.923831 5495 ts_tablet_manager.cc:1428] T 12baa1a998374a229980bf6a1d02a950 P 5565ec2abbf846858fef0f8311f80171: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 15:08:07.180550 5503 raft_consensus.cc:482] T 12baa1a998374a229980bf6a1d02a950 P 5565ec2abbf846858fef0f8311f80171 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:08:07.180804 5503 raft_consensus.cc:504] T 12baa1a998374a229980bf6a1d02a950 P 5565ec2abbf846858fef0f8311f80171 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } }
I20260809 15:08:07.182374 5503 leader_election.cc:288] T 12baa1a998374a229980bf6a1d02a950 P 5565ec2abbf846858fef0f8311f80171 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32c5ad23e4204495a9f41f9a7dd668ad (127.24.183.193:39465), e17f925643d44003bf98d29f94a48a4b (127.24.183.194:46433)
I20260809 15:08:07.191128 5494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:07.182839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:52581 (local address 127.24.183.193:39465)
0809 15:08:07.183075 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:07.183083 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:07.184049 (+ 966us) server_negotiation.cc:407] Connection header received
0809 15:08:07.184172 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:07.184177 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:07.184272 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:07.184345 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:08:07.184972 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:07.185966 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:07.188690 (+ 2724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:07.189989 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:07.190157 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:07.190242 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:07.190717 (+ 475us) server_negotiation.cc:299] Negotiation successful
0809 15:08:07.190865 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 15:08:07.191165 5621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:07.182686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:46433 (local address 127.24.183.195:51741)
0809 15:08:07.183441 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:07.183464 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:07.183632 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:07.184138 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:07.184143 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:07.184178 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:08:07.184457 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:07.184470 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:07.186088 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:07.186093 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:07.188552 (+ 2459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:07.188568 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:07.190222 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:07.190227 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:07.190456 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:07.190530 (+ 74us) client_negotiation.cc:769] Sending connection context
0809 15:08:07.190613 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 15:08:07.190752 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"mutex_wait_us":122,"thread_start_us":1006,"threads_started":1}
I20260809 15:08:07.191522 5623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:07.182982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:51741 (local address 127.24.183.194:46433)
0809 15:08:07.183808 (+ 826us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:07.183814 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:07.183836 (+ 22us) server_negotiation.cc:407] Connection header received
0809 15:08:07.183913 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:07.183917 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:07.183985 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:07.184095 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:08:07.184688 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:07.185930 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:07.188696 (+ 2766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:07.190029 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:07.190159 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:07.190239 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:07.190693 (+ 454us) server_negotiation.cc:299] Negotiation successful
0809 15:08:07.190948 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":442,"threads_started":1}
I20260809 15:08:07.192116 5620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:07.182758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:39465 (local address 127.24.183.195:52581)
0809 15:08:07.183944 (+ 1186us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:07.183958 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:07.184059 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:07.184394 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:07.184398 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:07.184424 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:08:07.184812 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:07.184822 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:07.186089 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:07.186093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:07.188551 (+ 2458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:07.188568 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:07.190215 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:07.190222 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:07.190477 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:07.190519 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 15:08:07.191590 (+ 1071us) client_negotiation.cc:240] Negotiation successful
0809 15:08:07.191750 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1089,"mutex_wait_us":176,"thread_start_us":301,"threads_started":1}
I20260809 15:08:07.192240 5139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12baa1a998374a229980bf6a1d02a950" candidate_uuid: "5565ec2abbf846858fef0f8311f80171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17f925643d44003bf98d29f94a48a4b" is_pre_election: true
I20260809 15:08:07.192533 5139 raft_consensus.cc:2454] T 12baa1a998374a229980bf6a1d02a950 P e17f925643d44003bf98d29f94a48a4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5565ec2abbf846858fef0f8311f80171 in term 0.
I20260809 15:08:07.192955 4910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12baa1a998374a229980bf6a1d02a950" candidate_uuid: "5565ec2abbf846858fef0f8311f80171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" is_pre_election: true
I20260809 15:08:07.193042 5257 leader_election.cc:302] T 12baa1a998374a229980bf6a1d02a950 P 5565ec2abbf846858fef0f8311f80171 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5565ec2abbf846858fef0f8311f80171, e17f925643d44003bf98d29f94a48a4b; no voters:
I20260809 15:08:07.193298 4910 raft_consensus.cc:2454] T 12baa1a998374a229980bf6a1d02a950 P 32c5ad23e4204495a9f41f9a7dd668ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5565ec2abbf846858fef0f8311f80171 in term 0.
I20260809 15:08:07.193457 5503 raft_consensus.cc:2781] T 12baa1a998374a229980bf6a1d02a950 P 5565ec2abbf846858fef0f8311f80171 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:08:07.193583 5503 raft_consensus.cc:482] T 12baa1a998374a229980bf6a1d02a950 P 5565ec2abbf846858fef0f8311f80171 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:08:07.193694 5503 raft_consensus.cc:3037] T 12baa1a998374a229980bf6a1d02a950 P 5565ec2abbf846858fef0f8311f80171 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:08:07.196002 5503 raft_consensus.cc:504] T 12baa1a998374a229980bf6a1d02a950 P 5565ec2abbf846858fef0f8311f80171 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } }
I20260809 15:08:07.196583 5503 leader_election.cc:288] T 12baa1a998374a229980bf6a1d02a950 P 5565ec2abbf846858fef0f8311f80171 [CANDIDATE]: Term 1 election: Requested vote from peers 32c5ad23e4204495a9f41f9a7dd668ad (127.24.183.193:39465), e17f925643d44003bf98d29f94a48a4b (127.24.183.194:46433)
I20260809 15:08:07.197022 4910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12baa1a998374a229980bf6a1d02a950" candidate_uuid: "5565ec2abbf846858fef0f8311f80171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32c5ad23e4204495a9f41f9a7dd668ad"
I20260809 15:08:07.197073 5139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12baa1a998374a229980bf6a1d02a950" candidate_uuid: "5565ec2abbf846858fef0f8311f80171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17f925643d44003bf98d29f94a48a4b"
I20260809 15:08:07.197216 4910 raft_consensus.cc:3037] T 12baa1a998374a229980bf6a1d02a950 P 32c5ad23e4204495a9f41f9a7dd668ad [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:08:07.197273 5139 raft_consensus.cc:3037] T 12baa1a998374a229980bf6a1d02a950 P e17f925643d44003bf98d29f94a48a4b [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:08:07.199390 4910 raft_consensus.cc:2454] T 12baa1a998374a229980bf6a1d02a950 P 32c5ad23e4204495a9f41f9a7dd668ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5565ec2abbf846858fef0f8311f80171 in term 1.
I20260809 15:08:07.199389 5139 raft_consensus.cc:2454] T 12baa1a998374a229980bf6a1d02a950 P e17f925643d44003bf98d29f94a48a4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5565ec2abbf846858fef0f8311f80171 in term 1.
I20260809 15:08:07.199796 5257 leader_election.cc:302] T 12baa1a998374a229980bf6a1d02a950 P 5565ec2abbf846858fef0f8311f80171 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 32c5ad23e4204495a9f41f9a7dd668ad, 5565ec2abbf846858fef0f8311f80171, e17f925643d44003bf98d29f94a48a4b; no voters:
I20260809 15:08:07.200119 5503 raft_consensus.cc:2781] T 12baa1a998374a229980bf6a1d02a950 P 5565ec2abbf846858fef0f8311f80171 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:08:07.200541 5503 raft_consensus.cc:686] T 12baa1a998374a229980bf6a1d02a950 P 5565ec2abbf846858fef0f8311f80171 [term 1 LEADER]: Becoming Leader. State: Replica: 5565ec2abbf846858fef0f8311f80171, State: Running, Role: LEADER
I20260809 15:08:07.201164 5503 consensus_queue.cc:237] T 12baa1a998374a229980bf6a1d02a950 P 5565ec2abbf846858fef0f8311f80171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } }
I20260809 15:08:07.204708 4627 catalog_manager.cc:5495] T 12baa1a998374a229980bf6a1d02a950 P 5565ec2abbf846858fef0f8311f80171 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5565ec2abbf846858fef0f8311f80171 (127.24.183.195). New cstate: current_term: 1 leader_uuid: "5565ec2abbf846858fef0f8311f80171" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5565ec2abbf846858fef0f8311f80171" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36621 } health_report { overall_health: HEALTHY } } }
I20260809 15:08:07.291489 5628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:07.285781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60600 (local address 127.24.183.195:36621)
0809 15:08:07.286377 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:07.286381 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:07.286401 (+ 20us) server_negotiation.cc:407] Connection header received
0809 15:08:07.286493 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:07.286496 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:07.286575 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:07.286716 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:08:07.287075 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:07.287909 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:07.289161 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:07.289640 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:07.289745 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:07.290193 (+ 448us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:08:07.290832 (+ 639us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:08:07.290940 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:07.291062 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 15:08:07.291130 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":356,"threads_started":1}
I20260809 15:08:07.306391 5494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:07.301007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:37001 (local address 127.24.183.193:39465)
0809 15:08:07.301185 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:07.301192 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:07.301686 (+ 494us) server_negotiation.cc:407] Connection header received
0809 15:08:07.301864 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:07.301868 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:07.301940 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:07.302013 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:08:07.302505 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:07.303233 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:07.304649 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:07.305131 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:07.305242 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:07.305663 (+ 421us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:08:07.305781 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:08:07.305857 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:07.305986 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 15:08:07.306074 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 15:08:07.306476 5636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:07.300990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:39465 (local address 127.24.183.195:37001)
0809 15:08:07.301580 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:07.301596 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:07.301733 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:07.302041 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:07.302045 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:07.302056 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 15:08:07.302336 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:07.302345 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:07.303367 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:07.303371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:07.304511 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:07.304524 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:07.305304 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:07.305310 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:07.305433 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:07.305486 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 15:08:07.305874 (+ 388us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 15:08:07.305886 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 15:08:07.305962 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 15:08:07.306105 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":289,"threads_started":1}
I20260809 15:08:07.314631 5139 raft_consensus.cc:1261] T 12baa1a998374a229980bf6a1d02a950 P e17f925643d44003bf98d29f94a48a4b [term 1 FOLLOWER]: Refusing update from remote peer 5565ec2abbf846858fef0f8311f80171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:08:07.314845 4910 raft_consensus.cc:1261] T 12baa1a998374a229980bf6a1d02a950 P 32c5ad23e4204495a9f41f9a7dd668ad [term 1 FOLLOWER]: Refusing update from remote peer 5565ec2abbf846858fef0f8311f80171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:08:07.315234 5624 consensus_queue.cc:1035] T 12baa1a998374a229980bf6a1d02a950 P 5565ec2abbf846858fef0f8311f80171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e17f925643d44003bf98d29f94a48a4b" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:08:07.315802 5503 consensus_queue.cc:1035] T 12baa1a998374a229980bf6a1d02a950 P 5565ec2abbf846858fef0f8311f80171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32c5ad23e4204495a9f41f9a7dd668ad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:08:07.343852 5628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:07.337129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:45655 (local address 127.24.183.195:36621)
0809 15:08:07.337317 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:07.337331 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:07.337650 (+ 319us) server_negotiation.cc:407] Connection header received
0809 15:08:07.337729 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:07.337732 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:07.337793 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:07.337852 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:08:07.338424 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:07.339448 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:07.342253 (+ 2805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:07.342737 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:07.342839 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:07.343241 (+ 402us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:08:07.343367 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:08:07.343446 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:07.343539 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 15:08:07.343607 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 15:08:07.344058 5642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:07.336855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:36621 (local address 127.24.183.193:45655)
0809 15:08:07.337400 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:07.337421 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:07.337567 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:07.337922 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:07.337926 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:07.337938 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 15:08:07.338267 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:07.338278 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:07.339810 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:07.339827 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:07.342115 (+ 2288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:07.342129 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:07.342927 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:07.342932 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:07.343067 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:07.343111 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 15:08:07.343454 (+ 343us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 15:08:07.343464 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 15:08:07.343523 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 15:08:07.343642 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":330,"threads_started":1}
I20260809 15:08:07.378940 4627 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:08:07.379242 4627 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:08:07.382463 4627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 36c2e1a7410f4f569d153b1f1b6cc236: Sending DeleteTablet for 3 replicas of tablet 12baa1a998374a229980bf6a1d02a950
I20260809 15:08:07.383496 5098 tablet_service.cc:1514] Processing DeleteTablet for tablet 12baa1a998374a229980bf6a1d02a950 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:08:07 UTC) from {username='oryx', principal='oryx/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:52834
I20260809 15:08:07.383829 5532 tablet_replica.cc:330] T 12baa1a998374a229980bf6a1d02a950 P e17f925643d44003bf98d29f94a48a4b: stopping tablet replica
I20260809 15:08:07.384083 5532 raft_consensus.cc:2229] T 12baa1a998374a229980bf6a1d02a950 P e17f925643d44003bf98d29f94a48a4b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:08:07.384289 5532 raft_consensus.cc:2258] T 12baa1a998374a229980bf6a1d02a950 P e17f925643d44003bf98d29f94a48a4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:08:07.384856 5332 tablet_service.cc:1514] Processing DeleteTablet for tablet 12baa1a998374a229980bf6a1d02a950 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:08:07 UTC) from {username='oryx', principal='oryx/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:60588
I20260809 15:08:07.384876 4869 tablet_service.cc:1514] Processing DeleteTablet for tablet 12baa1a998374a229980bf6a1d02a950 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:08:07 UTC) from {username='oryx', principal='oryx/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:40770
I20260809 15:08:07.385171 5535 tablet_replica.cc:330] T 12baa1a998374a229980bf6a1d02a950 P 32c5ad23e4204495a9f41f9a7dd668ad: stopping tablet replica
I20260809 15:08:07.385180 5533 tablet_replica.cc:330] T 12baa1a998374a229980bf6a1d02a950 P 5565ec2abbf846858fef0f8311f80171: stopping tablet replica
I20260809 15:08:07.385370 5535 raft_consensus.cc:2229] T 12baa1a998374a229980bf6a1d02a950 P 32c5ad23e4204495a9f41f9a7dd668ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:08:07.385377 5533 raft_consensus.cc:2229] T 12baa1a998374a229980bf6a1d02a950 P 5565ec2abbf846858fef0f8311f80171 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:08:07.385560 5535 raft_consensus.cc:2258] T 12baa1a998374a229980bf6a1d02a950 P 32c5ad23e4204495a9f41f9a7dd668ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:08:07.385883 5533 raft_consensus.cc:2258] T 12baa1a998374a229980bf6a1d02a950 P 5565ec2abbf846858fef0f8311f80171 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:08:07.386757 5535 ts_tablet_manager.cc:1905] T 12baa1a998374a229980bf6a1d02a950 P 32c5ad23e4204495a9f41f9a7dd668ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:08:07.386757 5533 ts_tablet_manager.cc:1905] T 12baa1a998374a229980bf6a1d02a950 P 5565ec2abbf846858fef0f8311f80171: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:08:07.386804 5532 ts_tablet_manager.cc:1905] T 12baa1a998374a229980bf6a1d02a950 P e17f925643d44003bf98d29f94a48a4b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:08:07.391458 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 4754
I20260809 15:08:07.394174 5535 ts_tablet_manager.cc:1918] T 12baa1a998374a229980bf6a1d02a950 P 32c5ad23e4204495a9f41f9a7dd668ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:08:07.394331 5533 ts_tablet_manager.cc:1918] T 12baa1a998374a229980bf6a1d02a950 P 5565ec2abbf846858fef0f8311f80171: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:08:07.394497 5533 log.cc:1196] T 12baa1a998374a229980bf6a1d02a950 P 5565ec2abbf846858fef0f8311f80171: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-2/wal/wals/12baa1a998374a229980bf6a1d02a950
I20260809 15:08:07.394518 5535 log.cc:1196] T 12baa1a998374a229980bf6a1d02a950 P 32c5ad23e4204495a9f41f9a7dd668ad: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-0/wal/wals/12baa1a998374a229980bf6a1d02a950
I20260809 15:08:07.394955 5533 ts_tablet_manager.cc:1939] T 12baa1a998374a229980bf6a1d02a950 P 5565ec2abbf846858fef0f8311f80171: Deleting consensus metadata
I20260809 15:08:07.395113 5535 ts_tablet_manager.cc:1939] T 12baa1a998374a229980bf6a1d02a950 P 32c5ad23e4204495a9f41f9a7dd668ad: Deleting consensus metadata
I20260809 15:08:07.396266 4605 catalog_manager.cc:4841] TS 32c5ad23e4204495a9f41f9a7dd668ad (127.24.183.193:39465): tablet 12baa1a998374a229980bf6a1d02a950 (table test-table [id=73a4b295c70a408881caa19588cae545]) successfully deleted
I20260809 15:08:07.396632 4605 catalog_manager.cc:4841] TS 5565ec2abbf846858fef0f8311f80171 (127.24.183.195:36621): tablet 12baa1a998374a229980bf6a1d02a950 (table test-table [id=73a4b295c70a408881caa19588cae545]) successfully deleted
I20260809 15:08:07.400254 5532 ts_tablet_manager.cc:1918] T 12baa1a998374a229980bf6a1d02a950 P e17f925643d44003bf98d29f94a48a4b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:08:07.400434 5532 log.cc:1196] T 12baa1a998374a229980bf6a1d02a950 P e17f925643d44003bf98d29f94a48a4b: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288021313853-25311-0/minicluster-data/ts-1/wal/wals/12baa1a998374a229980bf6a1d02a950
I20260809 15:08:07.400843 5532 ts_tablet_manager.cc:1939] T 12baa1a998374a229980bf6a1d02a950 P e17f925643d44003bf98d29f94a48a4b: Deleting consensus metadata
I20260809 15:08:07.402077 4603 catalog_manager.cc:4841] TS e17f925643d44003bf98d29f94a48a4b (127.24.183.194:46433): tablet 12baa1a998374a229980bf6a1d02a950 (table test-table [id=73a4b295c70a408881caa19588cae545]) successfully deleted
Aug 09 15:08:07 dist-test-slave-nj21 krb5kdc[4555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288086, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.183.193@KRBTEST.COM
I20260809 15:08:07.411475 5494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:07.393237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40802 (local address 127.24.183.193:39465)
0809 15:08:07.393418 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:07.393425 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:07.396957 (+ 3532us) server_negotiation.cc:407] Connection header received
0809 15:08:07.397185 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:07.397190 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:07.397265 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:07.397349 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:07.398582 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:07.399258 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:07.400639 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:07.401085 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:07.401199 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:07.405637 (+ 4438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:07.405653 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:07.405671 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:08:07.405706 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:07.408670 (+ 2964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:07.409351 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:07.409358 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:07.409361 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:07.409512 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:07.409886 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:07.409889 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:07.409891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:07.410271 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:07.410597 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:07.410906 (+ 309us) server_negotiation.cc:299] Negotiation successful
0809 15:08:07.411148 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 15:08:07.412206 4974 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:08:07.570005 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 4754
I20260809 15:08:07.603807 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 4984
Aug 09 15:08:07 dist-test-slave-nj21 krb5kdc[4555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288086, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.183.194@KRBTEST.COM
I20260809 15:08:07.620106 5623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:07.604575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52838 (local address 127.24.183.194:46433)
0809 15:08:07.604803 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:07.604809 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:07.604830 (+ 21us) server_negotiation.cc:407] Connection header received
0809 15:08:07.605016 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:07.605021 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:07.605090 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:07.605174 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:07.606393 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:07.607116 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:07.608459 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:07.608876 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:07.608987 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:07.613356 (+ 4369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:07.613375 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:07.613393 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:08:07.613449 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:07.617205 (+ 3756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:07.617885 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:07.617895 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:07.617899 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:07.618053 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:07.618384 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:07.618408 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:07.618412 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:07.618902 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:07.619214 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:07.619447 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 15:08:07.619744 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 15:08:07.620802 5212 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:08:07.742463 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 4984
I20260809 15:08:07.768086 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 5226
Aug 09 15:08:07 dist-test-slave-nj21 krb5kdc[4555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288086, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.183.195@KRBTEST.COM
I20260809 15:08:07.784926 5628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:07.768858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60616 (local address 127.24.183.195:36621)
0809 15:08:07.769084 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:07.769090 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:07.769117 (+ 27us) server_negotiation.cc:407] Connection header received
0809 15:08:07.769188 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:07.769194 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:07.769306 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:07.769452 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:07.770774 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:07.771958 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:07.773193 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:07.773886 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:07.774017 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:07.778520 (+ 4503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:07.778538 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:07.778559 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:08:07.778603 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:07.782106 (+ 3503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:07.782766 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:07.782775 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:07.782778 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:07.782925 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:07.783291 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:07.783294 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:07.783297 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:07.783745 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:07.784080 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:07.784378 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 15:08:07.784620 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 15:08:07.785633 5444 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:08:07.933655 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 5226
I20260809 15:08:07.962033 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 4571
I20260809 15:08:07.963376 4709 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:08:08.085502 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 4571
2026-08-09T15:08:08Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6150 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:08:08 dist-test-slave-nj21 krb5kdc[5657](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:08:08 dist-test-slave-nj21 krb5kdc[5657](info): set up 2 sockets
Aug 09 15:08:08 dist-test-slave-nj21 krb5kdc[5657](info): commencing operation
krb5kdc: starting...
W20260809 15:08:09.322854 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.004s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:08:09 dist-test-slave-nj21 krb5kdc[5657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:08:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:08:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
I20260809 15:08:09.570533 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:43439
--webserver_interface=127.24.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:36567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:43439
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:08:09.898888 5679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:08:09.899191 5679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:08:09.899340 5679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:08:09.906533 5679 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:08:09.906648 5679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:08:09.906739 5679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:08:09.906818 5679 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:08:09.906884 5679 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:08:09.916711 5679 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:36567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:43439
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:43439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:08:09.917977 5679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:08:09.919267 5679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:08:09.930547 5697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:09.930624 5689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:09.930593 5693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:08:09.931571 5679 server_base.cc:1034] running on GCE node
I20260809 15:08:09.932304 5679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:08:09.933688 5679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:08:09.934892 5679 hybrid_clock.cc:648] HybridClock initialized: now 1786288089934841 us; error 35 us; skew 500 ppm
I20260809 15:08:09.935225 5679 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T15:08:10Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1915 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:08:10 dist-test-slave-nj21 krb5kdc[5704](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:08:10 dist-test-slave-nj21 krb5kdc[5704](info): set up 2 sockets
Aug 09 15:08:10 dist-test-slave-nj21 krb5kdc[5704](info): commencing operation
krb5kdc: starting...
W20260809 15:08:11.224993 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.007s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:08:11 dist-test-slave-nj21 krb5kdc[5704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:08:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:08:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
I20260809 15:08:11.476702 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:39163
--webserver_interface=127.24.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:39175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:39163
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:08:11.805790 5729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:08:11.806061 5729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:08:11.806198 5729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:08:11.813231 5729 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:08:11.813337 5729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:08:11.813418 5729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:08:11.813498 5729 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:08:11.813556 5729 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:08:11.822973 5729 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:39175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:39163
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:39163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:08:11.824318 5729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:08:11.825632 5729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:08:11.836896 5740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:11.836944 5739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:11.836973 5744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:08:11.837900 5729 server_base.cc:1034] running on GCE node
I20260809 15:08:11.838590 5729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:08:11.839926 5729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:08:11.841122 5729 hybrid_clock.cc:648] HybridClock initialized: now 1786288091841077 us; error 33 us; skew 500 ppm
I20260809 15:08:11.841441 5729 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:08:11 dist-test-slave-nj21 krb5kdc[5704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288091, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:08:11.848028 5729 init.cc:377] Logged in from keytab as kudu/127.24.183.254@KRBTEST.COM (short username kudu)
I20260809 15:08:11.849470 5729 webserver.cc:460] Webserver started at http://127.24.183.254:45587/ using document root <none> and password file <none>
I20260809 15:08:11.850157 5729 fs_manager.cc:359] Metadata directory not provided
I20260809 15:08:11.850274 5729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:08:11.850585 5729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:08:11.853587 5729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "653fefb4627d4f8191a009e4b766dce7"
format_stamp: "Formatted at 2026-08-09 15:08:11 on dist-test-slave-nj21"
server_key: "04018882bc3ec9cbdda487844230e6f4"
server_key_iv: "cdfbb00026229d777e410b53f006b1d1"
server_key_version: "encryptionkey@0"
I20260809 15:08:11.854334 5729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "653fefb4627d4f8191a009e4b766dce7"
format_stamp: "Formatted at 2026-08-09 15:08:11 on dist-test-slave-nj21"
server_key: "04018882bc3ec9cbdda487844230e6f4"
server_key_iv: "cdfbb00026229d777e410b53f006b1d1"
server_key_version: "encryptionkey@0"
I20260809 15:08:11.859411 5729 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260809 15:08:11.862715 5759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:11.863997 5729 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:08:11.864149 5729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "653fefb4627d4f8191a009e4b766dce7"
format_stamp: "Formatted at 2026-08-09 15:08:11 on dist-test-slave-nj21"
server_key: "04018882bc3ec9cbdda487844230e6f4"
server_key_iv: "cdfbb00026229d777e410b53f006b1d1"
server_key_version: "encryptionkey@0"
I20260809 15:08:11.864319 5729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:08:11.888381 5729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:08:11.892140 5729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:08:11.892400 5729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:08:11.909513 5729 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:39163
I20260809 15:08:11.909545 5845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:39163 every 8 connection(s)
I20260809 15:08:11.910864 5729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:08:11.914585 5847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:08:11.917349 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 5729
I20260809 15:08:11.917567 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:08:11.918386 25311 external_mini_cluster.cc:1428] Setting key 2e2ba2a89614e3e1f78eadae681accde
I20260809 15:08:11.927448 5847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7: Bootstrap starting.
I20260809 15:08:11.931154 5847 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7: Neither blocks nor log segments found. Creating new log.
I20260809 15:08:11.932480 5847 log.cc:824] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7: Log is configured to *not* fsync() on all Append() calls
Aug 09 15:08:11 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:08:11.935727 5847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7: No bootstrap required, opened a new log
I20260809 15:08:11.943193 5852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:11.920563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48208 (local address 127.24.183.254:39163)
0809 15:08:11.921307 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:11.921318 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:11.921361 (+ 43us) server_negotiation.cc:407] Connection header received
0809 15:08:11.923463 (+ 2102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:11.923487 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:11.923909 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:11.924222 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:11.926044 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:11.927326 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:11.929081 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:11.929834 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:11.929967 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:11.934765 (+ 4798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:11.934777 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:11.934795 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:08:11.934840 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:11.938728 (+ 3888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:11.939397 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:11.939406 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:11.939415 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:11.939629 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:11.940030 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:11.940035 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:11.940039 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:11.940714 (+ 675us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:11.941032 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:11.941342 (+ 310us) server_negotiation.cc:299] Negotiation successful
0809 15:08:11.941786 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":344,"threads_started":1}
I20260809 15:08:11.945761 5847 raft_consensus.cc:348] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "653fefb4627d4f8191a009e4b766dce7" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 39163 } }
I20260809 15:08:11.946092 5847 raft_consensus.cc:374] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:08:11.946164 5847 raft_consensus.cc:729] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 653fefb4627d4f8191a009e4b766dce7, State: Initialized, Role: FOLLOWER
I20260809 15:08:11.946698 5847 consensus_queue.cc:260] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "653fefb4627d4f8191a009e4b766dce7" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 39163 } }
I20260809 15:08:11.946918 5847 raft_consensus.cc:388] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:08:11.947019 5847 raft_consensus.cc:482] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:08:11.947145 5847 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:08:11.949309 5847 raft_consensus.cc:504] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "653fefb4627d4f8191a009e4b766dce7" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 39163 } }
I20260809 15:08:11.949780 5847 leader_election.cc:302] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 653fefb4627d4f8191a009e4b766dce7; no voters:
I20260809 15:08:11.950332 5847 leader_election.cc:288] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:08:11.950497 5855 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:08:11.951705 5855 raft_consensus.cc:686] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7 [term 1 LEADER]: Becoming Leader. State: Replica: 653fefb4627d4f8191a009e4b766dce7, State: Running, Role: LEADER
I20260809 15:08:11.952207 5855 consensus_queue.cc:237] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "653fefb4627d4f8191a009e4b766dce7" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 39163 } }
I20260809 15:08:11.952473 5847 sys_catalog.cc:564] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:08:11.954751 5859 sys_catalog.cc:455] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 653fefb4627d4f8191a009e4b766dce7. Latest consensus state: current_term: 1 leader_uuid: "653fefb4627d4f8191a009e4b766dce7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "653fefb4627d4f8191a009e4b766dce7" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 39163 } } }
I20260809 15:08:11.955016 5856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "653fefb4627d4f8191a009e4b766dce7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "653fefb4627d4f8191a009e4b766dce7" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 39163 } } }
I20260809 15:08:11.955225 5859 sys_catalog.cc:458] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7 [sys.catalog]: This master's current role is: LEADER
I20260809 15:08:11.955287 5856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7 [sys.catalog]: This master's current role is: LEADER
I20260809 15:08:11.957530 5872 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:08:11.961824 5872 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:08:11.970172 5872 catalog_manager.cc:1347] Generated new cluster ID: bdf8064744444e91aca8014828aa51ba
I20260809 15:08:11.970268 5872 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:08:11.984041 5872 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:08:11.985246 5872 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:08:11.994608 5872 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7: Generated new TSK 0
I20260809 15:08:11.995373 5872 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab.
Entry for principal kudu/127.24.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab.
I20260809 15:08:12.067046 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.193:0
--local_ip_for_outbound_sockets=127.24.183.193
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:39163
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:39175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:08:12.404095 5890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:08:12.404361 5890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:08:12.404500 5890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:08:12.411850 5890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:08:12.411953 5890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:08:12.412110 5890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.193
I20260809 15:08:12.421921 5890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:39175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:39163
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.193
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:08:12.423273 5890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:08:12.424824 5890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:08:12.437717 5898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:12.437788 5905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:12.437752 5902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:08:12.438774 5890 server_base.cc:1034] running on GCE node
I20260809 15:08:12.439378 5890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:08:12.440271 5890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:08:12.441478 5890 hybrid_clock.cc:648] HybridClock initialized: now 1786288092441426 us; error 43 us; skew 500 ppm
I20260809 15:08:12.441774 5890 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:08:12 dist-test-slave-nj21 krb5kdc[5704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288092, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:08:12.444859 5890 init.cc:377] Logged in from keytab as kudu/127.24.183.193@KRBTEST.COM (short username kudu)
I20260809 15:08:12.446240 5890 webserver.cc:460] Webserver started at http://127.24.183.193:35043/ using document root <none> and password file <none>
I20260809 15:08:12.446906 5890 fs_manager.cc:359] Metadata directory not provided
I20260809 15:08:12.447022 5890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:08:12.447325 5890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:08:12.450282 5890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-0/data/instance:
uuid: "469f740daf4347d19cb61e829560d687"
format_stamp: "Formatted at 2026-08-09 15:08:12 on dist-test-slave-nj21"
server_key: "d6b8f415e7688a19314cd06fcd56ad20"
server_key_iv: "17e60382a7d1ab4fb0c3cc387e7b8b69"
server_key_version: "encryptionkey@0"
I20260809 15:08:12.451050 5890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance:
uuid: "469f740daf4347d19cb61e829560d687"
format_stamp: "Formatted at 2026-08-09 15:08:12 on dist-test-slave-nj21"
server_key: "d6b8f415e7688a19314cd06fcd56ad20"
server_key_iv: "17e60382a7d1ab4fb0c3cc387e7b8b69"
server_key_version: "encryptionkey@0"
I20260809 15:08:12.456163 5890 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260809 15:08:12.459501 5916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:12.460750 5890 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:08:12.460892 5890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-0/wal
uuid: "469f740daf4347d19cb61e829560d687"
format_stamp: "Formatted at 2026-08-09 15:08:12 on dist-test-slave-nj21"
server_key: "d6b8f415e7688a19314cd06fcd56ad20"
server_key_iv: "17e60382a7d1ab4fb0c3cc387e7b8b69"
server_key_version: "encryptionkey@0"
I20260809 15:08:12.461058 5890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:08:12.478893 5890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:08:12.482491 5890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:08:12.482746 5890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:08:12.483630 5890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:08:12.484933 5890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:08:12.485010 5890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:12.485157 5890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:08:12.485210 5890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:12.514874 5890 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.193:39393
I20260809 15:08:12.514914 6110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.193:39393 every 8 connection(s)
I20260809 15:08:12.516108 5890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
I20260809 15:08:12.521847 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 5890
I20260809 15:08:12.522066 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance
I20260809 15:08:12.522706 25311 external_mini_cluster.cc:1428] Setting key fc92de3fcd42a0331b66fa45e77c870a
Aug 09 15:08:12 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288092, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:08:12.535925 6113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:12.518572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:48425 (local address 127.24.183.254:39163)
0809 15:08:12.518984 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:12.518989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:12.519948 (+ 959us) server_negotiation.cc:407] Connection header received
0809 15:08:12.521694 (+ 1746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:12.521701 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:12.521791 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:12.521922 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:12.523399 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:12.524254 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:12.525905 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:12.526501 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:12.526627 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:12.530009 (+ 3382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:12.530015 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:12.530019 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:08:12.530058 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:12.533034 (+ 2976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:12.533701 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:12.533706 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:12.533709 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:12.533854 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:12.534216 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:12.534220 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:12.534223 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:12.534567 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:12.534702 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:12.535310 (+ 608us) server_negotiation.cc:299] Negotiation successful
0809 15:08:12.535602 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":205,"threads_started":1}
I20260809 15:08:12.536938 6115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:12.519094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.193:48425)
0809 15:08:12.519800 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:12.519843 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:12.521439 (+ 1596us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:12.522136 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:12.522147 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:12.522649 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:12.523206 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:12.523236 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:12.524434 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:12.524440 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:12.525749 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:12.525763 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:12.526705 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:12.526710 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:12.526899 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:12.527502 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:12.527530 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:12.529796 (+ 2266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:08:12.533178 (+ 3382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:08:12.533185 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:08:12.533195 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:08:12.533593 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:08:12.533947 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:08:12.533951 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:08:12.533953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:08:12.534121 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:08:12.534671 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:08:12.534676 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:08:12.534960 (+ 284us) client_negotiation.cc:769] Sending connection context
0809 15:08:12.535172 (+ 212us) client_negotiation.cc:240] Negotiation successful
0809 15:08:12.535590 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":266,"threads_started":1}
I20260809 15:08:12.538367 6111 heartbeater.cc:347] Connected to a master server at 127.24.183.254:39163
I20260809 15:08:12.538651 6111 heartbeater.cc:464] Registering TS with master...
I20260809 15:08:12.539340 6111 heartbeater.cc:511] Master 127.24.183.254:39163 requested a full tablet report, sending...
I20260809 15:08:12.541414 5782 ts_manager.cc:194] Registered new tserver with Master: 469f740daf4347d19cb61e829560d687 (127.24.183.193:39393)
I20260809 15:08:12.542842 5782 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.193@KRBTEST.COM'} at 127.24.183.193:48425
WARNING: no policy specified for kudu/127.24.183.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab.
Entry for principal kudu/127.24.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab.
I20260809 15:08:12.578624 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.194:0
--local_ip_for_outbound_sockets=127.24.183.194
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:39163
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:39175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:08:12.913398 6118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:08:12.913702 6118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:08:12.913856 6118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:08:12.925040 6118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:08:12.925164 6118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:08:12.925341 6118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.194
I20260809 15:08:12.937064 6118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:39175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:39163
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.194
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:08:12.938211 6118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:08:12.939465 6118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:08:12.952003 6123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:12.952026 6127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:12.952142 6125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:08:12.953260 6118 server_base.cc:1034] running on GCE node
I20260809 15:08:12.953859 6118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:08:12.954735 6118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:08:12.955955 6118 hybrid_clock.cc:648] HybridClock initialized: now 1786288092955899 us; error 33 us; skew 500 ppm
I20260809 15:08:12.956262 6118 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:08:12 dist-test-slave-nj21 krb5kdc[5704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288092, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:08:12.959367 6118 init.cc:377] Logged in from keytab as kudu/127.24.183.194@KRBTEST.COM (short username kudu)
I20260809 15:08:12.960743 6118 webserver.cc:460] Webserver started at http://127.24.183.194:35965/ using document root <none> and password file <none>
I20260809 15:08:12.961356 6118 fs_manager.cc:359] Metadata directory not provided
I20260809 15:08:12.961469 6118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:08:12.961725 6118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:08:12.964712 6118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-1/data/instance:
uuid: "f781bf7bb7924cb59c9c52c3ef3b9704"
format_stamp: "Formatted at 2026-08-09 15:08:12 on dist-test-slave-nj21"
server_key: "997216aed9f349e803ba95f488e5c663"
server_key_iv: "2d51d5ac916779c9fdc6225aa1d90ba0"
server_key_version: "encryptionkey@0"
I20260809 15:08:12.965471 6118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance:
uuid: "f781bf7bb7924cb59c9c52c3ef3b9704"
format_stamp: "Formatted at 2026-08-09 15:08:12 on dist-test-slave-nj21"
server_key: "997216aed9f349e803ba95f488e5c663"
server_key_iv: "2d51d5ac916779c9fdc6225aa1d90ba0"
server_key_version: "encryptionkey@0"
I20260809 15:08:12.970631 6118 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 15:08:12.973971 6136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:12.975174 6118 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:08:12.975306 6118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-1/wal
uuid: "f781bf7bb7924cb59c9c52c3ef3b9704"
format_stamp: "Formatted at 2026-08-09 15:08:12 on dist-test-slave-nj21"
server_key: "997216aed9f349e803ba95f488e5c663"
server_key_iv: "2d51d5ac916779c9fdc6225aa1d90ba0"
server_key_version: "encryptionkey@0"
I20260809 15:08:12.975484 6118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:08:12.997974 6118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:08:13.001657 6118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:08:13.001883 6118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:08:13.002766 6118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:08:13.004051 6118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:08:13.004124 6118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:13.004216 6118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:08:13.004259 6118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:13.035156 6118 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.194:45717
I20260809 15:08:13.035192 6336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.194:45717 every 8 connection(s)
I20260809 15:08:13.036492 6118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
I20260809 15:08:13.041756 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 6118
I20260809 15:08:13.041913 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance
I20260809 15:08:13.042397 25311 external_mini_cluster.cc:1428] Setting key b3583c84f3d963c22990bfdea2cfec49
Aug 09 15:08:13 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288092, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:08:13.057029 6342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:13.039006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:44273 (local address 127.24.183.254:39163)
0809 15:08:13.039488 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:13.039495 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:13.040479 (+ 984us) server_negotiation.cc:407] Connection header received
0809 15:08:13.042303 (+ 1824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:13.042310 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:13.042408 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:13.042544 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:13.044494 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:13.045230 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:13.046739 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:13.047177 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:13.047290 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:13.051013 (+ 3723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:13.051019 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:13.051024 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:08:13.051062 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:13.053840 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:13.054623 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:13.054627 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:13.054629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:13.054785 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:13.055265 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:13.055268 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:13.055270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:13.055547 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:13.055693 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:13.056510 (+ 817us) server_negotiation.cc:299] Negotiation successful
0809 15:08:13.056682 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":218,"threads_started":1}
I20260809 15:08:13.058509 6348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:13.039550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.194:44273)
0809 15:08:13.040299 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:13.040348 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:13.042040 (+ 1692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:13.042973 (+ 933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:13.042984 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:13.043659 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:13.044326 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:13.044346 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:13.045382 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:13.045386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:13.046598 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:13.046609 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:13.047399 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:13.047406 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:13.047627 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:13.048272 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:13.048295 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:13.050787 (+ 2492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:08:13.053972 (+ 3185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:08:13.053981 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:08:13.053995 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:08:13.054518 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:08:13.054916 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:08:13.054921 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:08:13.054925 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:08:13.055163 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:08:13.055694 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:08:13.055702 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:08:13.056144 (+ 442us) client_negotiation.cc:769] Sending connection context
0809 15:08:13.056423 (+ 279us) client_negotiation.cc:240] Negotiation successful
0809 15:08:13.056951 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":293,"threads_started":1}
I20260809 15:08:13.059986 6339 heartbeater.cc:347] Connected to a master server at 127.24.183.254:39163
I20260809 15:08:13.060282 6339 heartbeater.cc:464] Registering TS with master...
I20260809 15:08:13.060989 6339 heartbeater.cc:511] Master 127.24.183.254:39163 requested a full tablet report, sending...
I20260809 15:08:13.062407 5782 ts_manager.cc:194] Registered new tserver with Master: f781bf7bb7924cb59c9c52c3ef3b9704 (127.24.183.194:45717)
I20260809 15:08:13.063227 5782 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.194@KRBTEST.COM'} at 127.24.183.194:44273
WARNING: no policy specified for kudu/127.24.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab.
Entry for principal kudu/127.24.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab.
I20260809 15:08:13.098587 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.195:0
--local_ip_for_outbound_sockets=127.24.183.195
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:39163
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:39175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:08:13.432587 6355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:08:13.432855 6355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:08:13.432951 6355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:08:13.440147 6355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:08:13.440249 6355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:08:13.440371 6355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.195
I20260809 15:08:13.449810 6355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:39175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:39163
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.195
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:08:13.450986 6355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:08:13.452214 6355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:08:13.464704 6362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:13.464735 6361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:13.464737 6366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:08:13.465603 6355 server_base.cc:1034] running on GCE node
I20260809 15:08:13.466157 6355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:08:13.467043 6355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:08:13.468240 6355 hybrid_clock.cc:648] HybridClock initialized: now 1786288093468180 us; error 47 us; skew 500 ppm
I20260809 15:08:13.468541 6355 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:08:13 dist-test-slave-nj21 krb5kdc[5704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288093, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:08:13.471640 6355 init.cc:377] Logged in from keytab as kudu/127.24.183.195@KRBTEST.COM (short username kudu)
I20260809 15:08:13.473094 6355 webserver.cc:460] Webserver started at http://127.24.183.195:45827/ using document root <none> and password file <none>
I20260809 15:08:13.473749 6355 fs_manager.cc:359] Metadata directory not provided
I20260809 15:08:13.473847 6355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:08:13.474108 6355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:08:13.476974 6355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-2/data/instance:
uuid: "90e34a25c1f841aea8aabe515e552e18"
format_stamp: "Formatted at 2026-08-09 15:08:13 on dist-test-slave-nj21"
server_key: "4a9a1d62d126de29a3f049cd22ee324b"
server_key_iv: "eecd3278c9f61b65b88b0a99bab36fe8"
server_key_version: "encryptionkey@0"
I20260809 15:08:13.477722 6355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance:
uuid: "90e34a25c1f841aea8aabe515e552e18"
format_stamp: "Formatted at 2026-08-09 15:08:13 on dist-test-slave-nj21"
server_key: "4a9a1d62d126de29a3f049cd22ee324b"
server_key_iv: "eecd3278c9f61b65b88b0a99bab36fe8"
server_key_version: "encryptionkey@0"
I20260809 15:08:13.482864 6355 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 15:08:13.486155 6377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:13.487401 6355 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:08:13.487560 6355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-2/wal
uuid: "90e34a25c1f841aea8aabe515e552e18"
format_stamp: "Formatted at 2026-08-09 15:08:13 on dist-test-slave-nj21"
server_key: "4a9a1d62d126de29a3f049cd22ee324b"
server_key_iv: "eecd3278c9f61b65b88b0a99bab36fe8"
server_key_version: "encryptionkey@0"
I20260809 15:08:13.487766 6355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:08:13.501634 6355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:08:13.505456 6355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:08:13.505725 6355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:08:13.506693 6355 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:08:13.508050 6355 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:08:13.508123 6355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:13.508237 6355 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:08:13.508293 6355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:13.539458 6355 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.195:35015
I20260809 15:08:13.539494 6557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.195:35015 every 8 connection(s)
I20260809 15:08:13.540745 6355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
I20260809 15:08:13.545643 6111 heartbeater.cc:503] Master 127.24.183.254:39163 was elected leader, sending a full tablet report...
I20260809 15:08:13.550532 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 6355
I20260809 15:08:13.550745 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance
I20260809 15:08:13.551218 25311 external_mini_cluster.cc:1428] Setting key 60b03748fb0cf40389da63e708c41861
Aug 09 15:08:13 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288093, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:08:13.560791 6342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:13.543211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:37165 (local address 127.24.183.254:39163)
0809 15:08:13.543439 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:13.543446 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:13.544614 (+ 1168us) server_negotiation.cc:407] Connection header received
0809 15:08:13.546501 (+ 1887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:13.546506 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:13.546607 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:13.546693 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:13.548414 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:13.549182 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:13.550965 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:13.551415 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:13.551519 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:13.555326 (+ 3807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:13.555331 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:13.555334 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:08:13.555366 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:13.557955 (+ 2589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:13.558586 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:13.558589 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:13.558591 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:13.558707 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:13.559098 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:13.559103 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:13.559107 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:13.559389 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:13.559537 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:13.560228 (+ 691us) server_negotiation.cc:299] Negotiation successful
0809 15:08:13.560461 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 15:08:13.561678 6563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:13.543715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.195:37165)
0809 15:08:13.544451 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:13.544488 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:13.546247 (+ 1759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:13.546931 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:13.546940 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:13.547625 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:13.548249 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:13.548266 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:13.549322 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:13.549326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:13.550817 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:13.550832 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:13.551646 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:13.551653 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:13.551907 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:13.552600 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:13.552629 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:13.555133 (+ 2504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:08:13.558074 (+ 2941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:08:13.558081 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:08:13.558092 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:08:13.558497 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:08:13.558779 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:08:13.558782 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:08:13.558784 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:08:13.558958 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:08:13.559516 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:08:13.559523 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:08:13.559874 (+ 351us) client_negotiation.cc:769] Sending connection context
0809 15:08:13.560104 (+ 230us) client_negotiation.cc:240] Negotiation successful
0809 15:08:13.560614 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":285,"threads_started":1}
I20260809 15:08:13.562981 6559 heartbeater.cc:347] Connected to a master server at 127.24.183.254:39163
I20260809 15:08:13.563282 6559 heartbeater.cc:464] Registering TS with master...
I20260809 15:08:13.564039 6559 heartbeater.cc:511] Master 127.24.183.254:39163 requested a full tablet report, sending...
I20260809 15:08:13.565665 5779 ts_manager.cc:194] Registered new tserver with Master: 90e34a25c1f841aea8aabe515e552e18 (127.24.183.195:35015)
I20260809 15:08:13.566584 25311 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:08:13.566802 5779 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.195@KRBTEST.COM'} at 127.24.183.195:37165
WARNING: no policy specified for oryx/127.24.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.193.keytab.
Entry for principal oryx/127.24.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.193.keytab.
WARNING: no policy specified for oryx/127.24.183.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.183.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.194.keytab.
Entry for principal oryx/127.24.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.194.keytab.
WARNING: no policy specified for oryx/127.24.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.195.keytab.
Entry for principal oryx/127.24.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.195.keytab.
I20260809 15:08:13.718605 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 5890
Aug 09 15:08:13 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.193@KRBTEST.COM
I20260809 15:08:13.734670 6572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:13.719911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36686 (local address 127.24.183.193:39393)
0809 15:08:13.720617 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:13.720624 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:13.720653 (+ 29us) server_negotiation.cc:407] Connection header received
0809 15:08:13.720717 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:13.720724 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:13.720959 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:13.721090 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:13.722850 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:13.723990 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:13.725122 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:13.725672 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:13.725768 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:13.729552 (+ 3784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:13.729564 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:13.729579 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:08:13.729612 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:13.732449 (+ 2837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:13.733025 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:13.733031 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:13.733033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:13.733160 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:13.733425 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:13.733429 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:13.733433 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:13.733757 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:13.734000 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:13.734181 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 15:08:13.734375 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":365,"threads_started":1}
I20260809 15:08:13.735980 6102 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:08:13.819779 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 5890
I20260809 15:08:13.840649 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 6118
Aug 09 15:08:13 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.194@KRBTEST.COM
I20260809 15:08:13.856724 6574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:13.841568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59790 (local address 127.24.183.194:45717)
0809 15:08:13.842110 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:13.842117 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:13.842144 (+ 27us) server_negotiation.cc:407] Connection header received
0809 15:08:13.842206 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:13.842212 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:13.842433 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:13.842579 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:13.843927 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:13.844968 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:13.846265 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:13.846820 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:13.846935 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:13.851095 (+ 4160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:13.851106 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:13.851122 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:08:13.851165 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:13.853943 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:13.854568 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:13.854575 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:13.854577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:13.854707 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:13.855070 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:13.855074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:13.855076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:13.855462 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:13.855941 (+ 479us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:13.856105 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 15:08:13.856368 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":264,"threads_started":1}
I20260809 15:08:13.857969 6329 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:08:13.940973 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 6118
I20260809 15:08:13.963304 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 6355
Aug 09 15:08:13 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.195@KRBTEST.COM
I20260809 15:08:13.979230 6577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:13.964306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50278 (local address 127.24.183.195:35015)
0809 15:08:13.964997 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:13.965005 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:13.965035 (+ 30us) server_negotiation.cc:407] Connection header received
0809 15:08:13.965097 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:13.965104 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:13.965320 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:13.965463 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:13.966850 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:13.968040 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:13.969409 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:13.969977 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:13.970093 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:13.974295 (+ 4202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:13.974307 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:13.974323 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:08:13.974359 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:13.976974 (+ 2615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:13.977598 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:13.977605 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:13.977607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:13.977727 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:13.978004 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:13.978007 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:13.978009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:13.978317 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:13.978572 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:13.978738 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 15:08:13.978934 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":351,"threads_started":1}
I20260809 15:08:13.980594 6552 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:08:14.063756 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 6355
I20260809 15:08:14.087426 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 5729
I20260809 15:08:14.088671 5837 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:08:14.194890 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 5729
I20260809 15:08:14.220952 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:39163
--webserver_interface=127.24.183.254
--webserver_port=45587
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:39175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:39163
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:08:14.566581 6582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:08:14.566880 6582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:08:14.567020 6582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:08:14.574152 6582 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:08:14.574265 6582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:08:14.574347 6582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:08:14.574425 6582 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:08:14.574492 6582 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:08:14.584108 6582 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:39175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:39163
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:39163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=45587
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:08:14.585441 6582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:08:14.586760 6582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:08:14.598016 6590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:14.598055 6591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:14.597978 6595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:08:14.599009 6582 server_base.cc:1034] running on GCE node
I20260809 15:08:14.599820 6582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:08:14.601197 6582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:08:14.602411 6582 hybrid_clock.cc:648] HybridClock initialized: now 1786288094602360 us; error 31 us; skew 500 ppm
I20260809 15:08:14.602749 6582 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:08:14 dist-test-slave-nj21 krb5kdc[5704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288094, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:08:14.605770 6582 init.cc:377] Logged in from keytab as kudu/127.24.183.254@KRBTEST.COM (short username kudu)
I20260809 15:08:14.607120 6582 webserver.cc:460] Webserver started at http://127.24.183.254:45587/ using document root <none> and password file <none>
I20260809 15:08:14.607791 6582 fs_manager.cc:359] Metadata directory not provided
I20260809 15:08:14.607915 6582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:08:14.613165 6582 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260809 15:08:14.615932 6610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:14.617215 6582 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:08:14.617360 6582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "653fefb4627d4f8191a009e4b766dce7"
format_stamp: "Formatted at 2026-08-09 15:08:11 on dist-test-slave-nj21"
server_key: "04018882bc3ec9cbdda487844230e6f4"
server_key_iv: "cdfbb00026229d777e410b53f006b1d1"
server_key_version: "encryptionkey@0"
I20260809 15:08:14.618491 6582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:08:14.649350 6582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:08:14.653604 6582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:08:14.653841 6582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:08:14.671855 6582 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:39163
I20260809 15:08:14.671878 6689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:39163 every 8 connection(s)
I20260809 15:08:14.673332 6582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:08:14.675129 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 6582
I20260809 15:08:14.675942 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.193:39393
--local_ip_for_outbound_sockets=127.24.183.193
--tserver_master_addrs=127.24.183.254:39163
--webserver_port=35043
--webserver_interface=127.24.183.193
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:39175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.183.193
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:08:14.680260 6690 sys_catalog.cc:263] Verifying existing consensus state
I20260809 15:08:14.683426 6690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7: Bootstrap starting.
I20260809 15:08:14.702798 6690 log.cc:824] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7: Log is configured to *not* fsync() on all Append() calls
I20260809 15:08:14.711701 6690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:08:14.712194 6690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7: Bootstrap complete.
I20260809 15:08:14.720999 6690 raft_consensus.cc:348] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "653fefb4627d4f8191a009e4b766dce7" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 39163 } }
I20260809 15:08:14.721894 6690 raft_consensus.cc:729] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 653fefb4627d4f8191a009e4b766dce7, State: Initialized, Role: FOLLOWER
I20260809 15:08:14.722476 6690 consensus_queue.cc:260] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "653fefb4627d4f8191a009e4b766dce7" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 39163 } }
I20260809 15:08:14.722728 6690 raft_consensus.cc:388] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:08:14.722836 6690 raft_consensus.cc:482] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:08:14.723013 6690 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:08:14.726684 6690 raft_consensus.cc:504] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "653fefb4627d4f8191a009e4b766dce7" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 39163 } }
I20260809 15:08:14.727133 6690 leader_election.cc:302] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 653fefb4627d4f8191a009e4b766dce7; no voters:
I20260809 15:08:14.727730 6690 leader_election.cc:288] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 15:08:14.727887 6697 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:08:14.728991 6697 raft_consensus.cc:686] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7 [term 2 LEADER]: Becoming Leader. State: Replica: 653fefb4627d4f8191a009e4b766dce7, State: Running, Role: LEADER
I20260809 15:08:14.729612 6697 consensus_queue.cc:237] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "653fefb4627d4f8191a009e4b766dce7" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 39163 } }
I20260809 15:08:14.729874 6690 sys_catalog.cc:564] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:08:14.732105 6700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 653fefb4627d4f8191a009e4b766dce7. Latest consensus state: current_term: 2 leader_uuid: "653fefb4627d4f8191a009e4b766dce7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "653fefb4627d4f8191a009e4b766dce7" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 39163 } } }
I20260809 15:08:14.732532 6700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7 [sys.catalog]: This master's current role is: LEADER
I20260809 15:08:14.733069 6698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "653fefb4627d4f8191a009e4b766dce7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "653fefb4627d4f8191a009e4b766dce7" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 39163 } } }
I20260809 15:08:14.733384 6698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7 [sys.catalog]: This master's current role is: LEADER
I20260809 15:08:14.744625 6707 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:08:14.751358 6707 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:08:14.752912 6707 catalog_manager.cc:1259] Loaded cluster ID: bdf8064744444e91aca8014828aa51ba
I20260809 15:08:14.753176 6707 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:08:14.757488 6707 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:08:14.759025 6707 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 653fefb4627d4f8191a009e4b766dce7: Loaded TSK: 0
I20260809 15:08:14.760679 6707 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 15:08:15.023008 6693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:08:15.023290 6693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:08:15.023454 6693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:08:15.030664 6693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:08:15.030802 6693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:08:15.030964 6693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.193
I20260809 15:08:15.040654 6693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:39175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.193.keytab
--principal=oryx/127.24.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.193:39393
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.183.193
--webserver_port=35043
--tserver_master_addrs=127.24.183.254:39163
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.193
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:08:15.042007 6693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:08:15.043311 6693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:08:15.058010 6740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:15.058041 6737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:15.058820 6735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:08:15.059383 6693 server_base.cc:1034] running on GCE node
I20260809 15:08:15.060093 6693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:08:15.061149 6693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:08:15.062393 6693 hybrid_clock.cc:648] HybridClock initialized: now 1786288095062340 us; error 34 us; skew 500 ppm
I20260809 15:08:15.062745 6693 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:08:15 dist-test-slave-nj21 krb5kdc[5704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288095, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:08:15.066576 6693 init.cc:377] Logged in from keytab as oryx/127.24.183.193@KRBTEST.COM (short username oryx)
I20260809 15:08:15.068281 6693 webserver.cc:460] Webserver started at http://127.24.183.193:35043/ using document root <none> and password file <none>
I20260809 15:08:15.069126 6693 fs_manager.cc:359] Metadata directory not provided
I20260809 15:08:15.069280 6693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:08:15.076220 6693 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 15:08:15.079419 6757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:15.081081 6693 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:08:15.081249 6693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-0/wal
uuid: "469f740daf4347d19cb61e829560d687"
format_stamp: "Formatted at 2026-08-09 15:08:12 on dist-test-slave-nj21"
server_key: "d6b8f415e7688a19314cd06fcd56ad20"
server_key_iv: "17e60382a7d1ab4fb0c3cc387e7b8b69"
server_key_version: "encryptionkey@0"
I20260809 15:08:15.082532 6693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:08:15.124675 6693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:08:15.128556 6693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:08:15.128794 6693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:08:15.129812 6693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:08:15.131302 6693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:08:15.131402 6693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:15.131517 6693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:08:15.131595 6693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:15.163194 6693 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.193:39393
I20260809 15:08:15.163267 6952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.193:39393 every 8 connection(s)
I20260809 15:08:15.164691 6693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
I20260809 15:08:15.173079 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 6693
I20260809 15:08:15.173687 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.194:45717
--local_ip_for_outbound_sockets=127.24.183.194
--tserver_master_addrs=127.24.183.254:39163
--webserver_port=35965
--webserver_interface=127.24.183.194
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:39175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.183.194
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 15:08:15 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:15 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:15.182749 6957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:15.167737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.193:49213)
0809 15:08:15.168598 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:15.168646 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:15.170359 (+ 1713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:15.171766 (+ 1407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:15.171777 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:15.172411 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:15.172949 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:15.172967 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:15.174660 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:15.174664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:15.175935 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:15.175948 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:15.176966 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:15.176974 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:15.177221 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:15.177850 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:15.177877 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:15.181492 (+ 3615us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":354,"threads_started":1}
W20260809 15:08:15.182961 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:15.167662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:49213 (local address 127.24.183.254:39163)
0809 15:08:15.168479 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:15.168494 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:15.168763 (+ 269us) server_negotiation.cc:407] Connection header received
0809 15:08:15.170905 (+ 2142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:15.170924 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:15.171235 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:15.171572 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:15.173139 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:15.174491 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:15.176072 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:15.176722 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:15.176901 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:15.181892 (+ 4991us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.193:49213: BlockingRecv error: recv got EOF from 127.24.183.193:49213 (error 108)
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":263,"threads_started":1}
W20260809 15:08:15.183872 6953 heartbeater.cc:644] Failed to heartbeat to 127.24.183.254:39163 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.183.254:39163: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Aug 09 15:08:15 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:15 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:15.193010 6957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:15.184471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.193:49783)
0809 15:08:15.184658 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:15.184677 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:15.184774 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:15.185086 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:15.185089 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:15.185472 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:15.185640 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:15.185649 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:15.186619 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:15.186624 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:15.187975 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:15.187989 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:15.188821 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:15.188828 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:15.189032 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:15.189618 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:15.189639 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:15.192756 (+ 3117us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 15:08:15.193466 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:15.184643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:49783 (local address 127.24.183.254:39163)
0809 15:08:15.184828 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:15.184834 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:15.184852 (+ 18us) server_negotiation.cc:407] Connection header received
0809 15:08:15.184901 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:15.184906 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:15.184973 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:15.185073 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:15.185761 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:15.186474 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:15.188117 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:15.188664 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:15.188778 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:15.192941 (+ 4163us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.193:49783: BlockingRecv error: recv got EOF from 127.24.183.193:49783 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 15:08:15 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:15 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:15.202673 6957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:15.193740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.193:59867)
0809 15:08:15.193962 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:15.193979 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:15.194066 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:15.194494 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:15.194499 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:15.194867 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:15.195050 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:15.195061 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:15.196435 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:15.196441 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:15.197916 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:15.197933 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:15.198794 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:15.198800 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:15.198988 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:15.199528 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:15.199551 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:15.202388 (+ 2837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125}
W20260809 15:08:15.202898 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:15.193844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:59867 (local address 127.24.183.254:39163)
0809 15:08:15.194165 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:15.194170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:15.194190 (+ 20us) server_negotiation.cc:407] Connection header received
0809 15:08:15.194241 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:15.194247 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:15.194340 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:15.194446 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:15.195180 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:15.196277 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:15.198050 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:15.198643 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:15.198746 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:15.202619 (+ 3873us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.193:59867: BlockingRecv error: recv got EOF from 127.24.183.193:59867 (error 108)
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":2017,"threads_started":1}
W20260809 15:08:15.203047 6953 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 15:08:15.513687 6958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:08:15.513967 6958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:08:15.514092 6958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:08:15.521207 6958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:08:15.521317 6958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:08:15.521476 6958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.194
I20260809 15:08:15.531046 6958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:39175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.194.keytab
--principal=oryx/127.24.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.194:45717
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.183.194
--webserver_port=35965
--tserver_master_addrs=127.24.183.254:39163
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.194
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:08:15.532368 6958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:08:15.533737 6958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:08:15.546384 6968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:15.546500 6969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:15.546474 6971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:08:15.547326 6958 server_base.cc:1034] running on GCE node
I20260809 15:08:15.547968 6958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:08:15.548858 6958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:08:15.550069 6958 hybrid_clock.cc:648] HybridClock initialized: now 1786288095550022 us; error 28 us; skew 500 ppm
I20260809 15:08:15.550379 6958 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:08:15 dist-test-slave-nj21 krb5kdc[5704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288095, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:08:15.553603 6958 init.cc:377] Logged in from keytab as oryx/127.24.183.194@KRBTEST.COM (short username oryx)
I20260809 15:08:15.554945 6958 webserver.cc:460] Webserver started at http://127.24.183.194:35965/ using document root <none> and password file <none>
I20260809 15:08:15.555599 6958 fs_manager.cc:359] Metadata directory not provided
I20260809 15:08:15.555748 6958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:08:15.561019 6958 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260809 15:08:15.563568 6988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:15.564841 6958 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 15:08:15.564993 6958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-1/wal
uuid: "f781bf7bb7924cb59c9c52c3ef3b9704"
format_stamp: "Formatted at 2026-08-09 15:08:12 on dist-test-slave-nj21"
server_key: "997216aed9f349e803ba95f488e5c663"
server_key_iv: "2d51d5ac916779c9fdc6225aa1d90ba0"
server_key_version: "encryptionkey@0"
I20260809 15:08:15.565928 6958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:08:15.587656 6958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:08:15.591666 6958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:08:15.591952 6958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:08:15.592897 6958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:08:15.594179 6958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:08:15.594282 6958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:15.594385 6958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:08:15.594492 6958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:15.624313 6958 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.194:45717
I20260809 15:08:15.624380 7169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.194:45717 every 8 connection(s)
I20260809 15:08:15.625753 6958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
I20260809 15:08:15.632757 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 6958
I20260809 15:08:15.633504 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.195:35015
--local_ip_for_outbound_sockets=127.24.183.195
--tserver_master_addrs=127.24.183.254:39163
--webserver_port=45827
--webserver_interface=127.24.183.195
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:39175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.183.195
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 15:08:15 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:15 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 15:08:15.642930 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:15.628235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:45959 (local address 127.24.183.254:39163)
0809 15:08:15.628439 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:15.628446 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:15.629658 (+ 1212us) server_negotiation.cc:407] Connection header received
0809 15:08:15.631558 (+ 1900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:15.631563 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:15.631637 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:15.631759 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:15.633604 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:15.634361 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:15.636549 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:15.637037 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:15.637162 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:15.642629 (+ 5467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.194:45959: BlockingRecv error: recv got EOF from 127.24.183.194:45959 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 15:08:15.643641 7172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:15.628765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.194:45959)
0809 15:08:15.629506 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:15.629546 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:15.631304 (+ 1758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:15.631951 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:15.631958 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:15.632628 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:15.633410 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:15.633450 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:15.634536 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:15.634541 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:15.636365 (+ 1824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:15.636381 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:15.637280 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:15.637287 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:15.637559 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:15.638267 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:15.638302 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:15.642385 (+ 4083us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":278,"threads_started":1}
W20260809 15:08:15.644484 7170 heartbeater.cc:644] Failed to heartbeat to 127.24.183.254:39163 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.183.254:39163: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Aug 09 15:08:15 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:15 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:15.653041 7172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:15.645124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.194:39851)
0809 15:08:15.645309 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:15.645330 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:15.645438 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:15.645814 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:15.645819 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:15.646159 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:15.646316 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:15.646326 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:15.647362 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:15.647368 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:15.648325 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:15.648339 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:15.649090 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:15.649096 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:15.649272 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:15.649842 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:15.649861 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:15.652714 (+ 2853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 15:08:15.653431 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:15.645278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:39851 (local address 127.24.183.254:39163)
0809 15:08:15.645449 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:15.645456 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:15.645476 (+ 20us) server_negotiation.cc:407] Connection header received
0809 15:08:15.645574 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:15.645580 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:15.645651 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:15.645753 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:15.646468 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:15.647215 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:15.648475 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:15.648929 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:15.649061 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:15.653093 (+ 4032us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.194:39851: BlockingRecv error: recv got EOF from 127.24.183.194:39851 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 09 15:08:15 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:15 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:15.663507 7172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:15.654266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.194:35241)
0809 15:08:15.654450 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:15.654472 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:15.654579 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:15.654940 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:15.654943 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:15.655230 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:15.655386 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:15.655395 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:15.656784 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:15.656789 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:15.657747 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:15.657759 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:15.658514 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:15.658520 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:15.658706 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:15.659262 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:15.659281 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:15.663213 (+ 3932us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 15:08:15.663817 7170 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 15:08:15.663604 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:15.654475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:35241 (local address 127.24.183.254:39163)
0809 15:08:15.654664 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:15.654670 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:15.654689 (+ 19us) server_negotiation.cc:407] Connection header received
0809 15:08:15.654741 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:15.654746 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:15.654819 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:15.654939 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:15.655524 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:15.656602 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:15.657896 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:15.658355 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:15.658472 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:15.663421 (+ 4949us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.194:35241: BlockingRecv error: recv got EOF from 127.24.183.194:35241 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260809 15:08:15.979694 7174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:08:15.979985 7174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:08:15.980091 7174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:08:15.987416 7174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:08:15.987545 7174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:08:15.987686 7174 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.195
I20260809 15:08:15.997574 7174 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:39175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/krb5kdc/oryx_127.24.183.195.keytab
--principal=oryx/127.24.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.195:35015
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.183.195
--webserver_port=45827
--tserver_master_addrs=127.24.183.254:39163
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.195
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:08:15.998844 7174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:08:16.000166 7174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:08:16.012681 7180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:16.012768 7181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:16.012795 7184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:08:16.013782 7174 server_base.cc:1034] running on GCE node
I20260809 15:08:16.014389 7174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:08:16.015357 7174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:08:16.016566 7174 hybrid_clock.cc:648] HybridClock initialized: now 1786288096016518 us; error 37 us; skew 500 ppm
I20260809 15:08:16.016875 7174 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:08:16 dist-test-slave-nj21 krb5kdc[5704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288096, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:08:16.019876 7174 init.cc:377] Logged in from keytab as oryx/127.24.183.195@KRBTEST.COM (short username oryx)
I20260809 15:08:16.021355 7174 webserver.cc:460] Webserver started at http://127.24.183.195:45827/ using document root <none> and password file <none>
I20260809 15:08:16.022056 7174 fs_manager.cc:359] Metadata directory not provided
I20260809 15:08:16.022207 7174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:08:16.027660 7174 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:08:16.030310 7195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:16.031657 7174 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:08:16.031854 7174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-2/wal
uuid: "90e34a25c1f841aea8aabe515e552e18"
format_stamp: "Formatted at 2026-08-09 15:08:13 on dist-test-slave-nj21"
server_key: "4a9a1d62d126de29a3f049cd22ee324b"
server_key_iv: "eecd3278c9f61b65b88b0a99bab36fe8"
server_key_version: "encryptionkey@0"
I20260809 15:08:16.032797 7174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:08:16.052861 7174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:08:16.056655 7174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:08:16.056887 7174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:08:16.057819 7174 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:08:16.059128 7174 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:08:16.059219 7174 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:16.059324 7174 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:08:16.059389 7174 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:16.092940 7174 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.195:35015
I20260809 15:08:16.092994 7376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.195:35015 every 8 connection(s)
I20260809 15:08:16.094355 7174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
I20260809 15:08:16.101083 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 7174
Aug 09 15:08:16 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:16 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 15:08:16.110857 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:16.096932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:42581 (local address 127.24.183.254:39163)
0809 15:08:16.097136 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:16.097143 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:16.098267 (+ 1124us) server_negotiation.cc:407] Connection header received
0809 15:08:16.100100 (+ 1833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:16.100104 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:16.100175 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:16.100264 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:16.101703 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:16.102617 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:16.104521 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:16.105145 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:16.105248 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:16.110657 (+ 5409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.195:42581: BlockingRecv error: recv got EOF from 127.24.183.195:42581 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 15:08:16.111649 7380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:16.097382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.195:42581)
0809 15:08:16.098103 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:16.098150 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:16.099838 (+ 1688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:16.100439 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:16.100446 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:16.100964 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:16.101531 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:16.101550 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:16.102813 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:16.102819 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:16.104372 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:16.104385 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:16.105371 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:16.105379 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:16.105658 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:16.106333 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:16.106363 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:16.110514 (+ 4151us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":262,"threads_started":1}
W20260809 15:08:16.112267 7377 heartbeater.cc:644] Failed to heartbeat to 127.24.183.254:39163 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.183.254:39163: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
I20260809 15:08:16.118005 7384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:16.102279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38414 (local address 127.24.183.254:39163)
0809 15:08:16.103211 (+ 932us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:16.103216 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:16.103233 (+ 17us) server_negotiation.cc:407] Connection header received
0809 15:08:16.103284 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:16.103288 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:16.103357 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:16.103479 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:16.105127 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:16.106175 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:16.107428 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:16.108035 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:16.108143 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:16.111333 (+ 3190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:16.111341 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:16.111355 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:08:16.111392 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:16.114414 (+ 3022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:16.115105 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:16.115114 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:16.115123 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:16.115381 (+ 258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:16.115741 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:16.115747 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:16.115750 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:16.116303 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:16.116539 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:16.117022 (+ 483us) server_negotiation.cc:299] Negotiation successful
0809 15:08:16.117621 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":824,"thread_start_us":682,"threads_started":1}
Aug 09 15:08:16 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:16 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:16.120733 7380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:16.112810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.195:35795)
0809 15:08:16.112986 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:16.113006 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:16.113087 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:16.113363 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:16.113368 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:16.113684 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:16.113808 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:16.113816 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:16.114816 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:16.114820 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:16.115988 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:16.116002 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:16.116828 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:16.116834 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:16.117020 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:16.117507 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:16.117529 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:16.120477 (+ 2948us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 15:08:16.120857 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:16.112914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:35795 (local address 127.24.183.254:39163)
0809 15:08:16.113043 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:16.113046 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:16.113061 (+ 15us) server_negotiation.cc:407] Connection header received
0809 15:08:16.113175 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:16.113178 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:16.113232 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:16.113312 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:16.113945 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:16.114689 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:16.116114 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:16.116665 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:16.116785 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:16.120655 (+ 3870us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.195:35795: BlockingRecv error: recv got EOF from 127.24.183.195:35795 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 09 15:08:16 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:16 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:16.127830 7380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:16.121333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.195:57077)
0809 15:08:16.121498 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:16.121515 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:16.121603 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:16.121899 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:16.121903 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:16.122183 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:16.122319 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:16.122326 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:16.123308 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:16.123311 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:16.124233 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:16.124244 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:16.124811 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:16.124815 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:16.124935 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:16.125334 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:16.125349 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:16.127592 (+ 2243us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":67}
W20260809 15:08:16.127976 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:16.121405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:57077 (local address 127.24.183.254:39163)
0809 15:08:16.121622 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:16.121627 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:16.121646 (+ 19us) server_negotiation.cc:407] Connection header received
0809 15:08:16.121687 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:16.121691 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:16.121764 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:16.121842 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:16.122454 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:16.123198 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:16.124336 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:16.124716 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:16.124800 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:16.127783 (+ 2983us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.195:57077: BlockingRecv error: recv got EOF from 127.24.183.195:57077 (error 108)
Metrics: {"server-negotiator.queue_time_us":122}
W20260809 15:08:16.128139 7377 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 15:08:16 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:16 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:16.212154 7386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:16.203699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.193:51135)
0809 15:08:16.204239 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:16.204259 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:16.204412 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:16.204762 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:16.204768 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:16.205273 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:16.205488 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:16.205498 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:16.206518 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:16.206524 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:16.207502 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:16.207515 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:16.208330 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:16.208337 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:16.208511 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:16.209098 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:16.209120 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:16.211913 (+ 2793us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":280,"threads_started":1}
W20260809 15:08:16.212321 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:16.203765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:51135 (local address 127.24.183.254:39163)
0809 15:08:16.203958 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:16.203965 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:16.204341 (+ 376us) server_negotiation.cc:407] Connection header received
0809 15:08:16.204560 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:16.204565 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:16.204632 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:16.204714 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:16.205636 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:16.206372 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:16.207647 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:16.208147 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:16.208260 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:16.212130 (+ 3870us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.193:51135: BlockingRecv error: recv got EOF from 127.24.183.193:51135 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 09 15:08:16 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:16 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:16.672582 7387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:16.664440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.194:50155)
0809 15:08:16.664905 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:16.664923 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:16.665045 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:16.665318 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:16.665321 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:16.665799 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:16.665999 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:16.666009 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:16.666929 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:16.666933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:16.667925 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:16.667938 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:16.668626 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:16.668629 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:16.668745 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:16.669289 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:16.669309 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:16.672255 (+ 2946us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":233,"threads_started":1}
W20260809 15:08:16.672703 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:16.664493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:50155 (local address 127.24.183.254:39163)
0809 15:08:16.664707 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:16.664713 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:16.664998 (+ 285us) server_negotiation.cc:407] Connection header received
0809 15:08:16.665137 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:16.665140 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:16.665205 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:16.665282 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:16.666111 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:16.666817 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:16.668045 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:16.668483 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:16.668579 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:16.672482 (+ 3903us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.194:50155: BlockingRecv error: recv got EOF from 127.24.183.194:50155 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 09 15:08:17 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:17 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:17.139834 7388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:17.128690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.195:52531)
0809 15:08:17.129151 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:17.129166 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:17.129276 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:17.129579 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:17.129583 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:17.130076 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:17.130275 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:17.130285 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:17.131383 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:17.131387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:17.132682 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:17.132700 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:17.133408 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:17.133413 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:17.133548 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:17.136618 (+ 3070us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:17.136638 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:17.139552 (+ 2914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":237,"threads_started":1}
W20260809 15:08:17.140019 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:17.128767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:52531 (local address 127.24.183.254:39163)
0809 15:08:17.128965 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:17.128972 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:17.129235 (+ 263us) server_negotiation.cc:407] Connection header received
0809 15:08:17.129366 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:17.129369 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:17.129459 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:17.129539 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:17.130407 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:17.131225 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:17.132817 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:17.133248 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:17.133348 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:17.139799 (+ 6451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.195:52531: BlockingRecv error: recv got EOF from 127.24.183.195:52531 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 15:08:17 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:17 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:17.221741 7389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:17.213096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.193:43915)
0809 15:08:17.213644 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:17.213665 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:17.213835 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:17.214176 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:17.214182 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:17.214627 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:17.214836 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:17.214847 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:17.215982 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:17.215987 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:17.216952 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:17.216966 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:17.217757 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:17.217766 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:17.217927 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:17.218492 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:17.218513 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:17.221366 (+ 2853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":377,"threads_started":1}
W20260809 15:08:17.221894 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:17.213236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:43915 (local address 127.24.183.254:39163)
0809 15:08:17.213459 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:17.213466 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:17.213759 (+ 293us) server_negotiation.cc:407] Connection header received
0809 15:08:17.213962 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:17.213967 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:17.214035 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:17.214127 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:17.214990 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:17.215830 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:17.217105 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:17.217567 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:17.217695 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:17.221606 (+ 3911us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.193:43915: BlockingRecv error: recv got EOF from 127.24.183.193:43915 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 15:08:17 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:17 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:17.684691 7390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:17.673835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.194:50657)
0809 15:08:17.674329 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:17.674345 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:17.674501 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:17.674879 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:17.674885 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:17.675434 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:17.675662 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:17.675695 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:17.677265 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:17.677271 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:17.678779 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:17.678796 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:17.679827 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:17.679833 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:17.680006 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:17.680880 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:17.680905 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:17.684440 (+ 3535us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":274,"threads_started":1}
W20260809 15:08:17.684952 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:17.673882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:50657 (local address 127.24.183.254:39163)
0809 15:08:17.674063 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:17.674069 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:17.674428 (+ 359us) server_negotiation.cc:407] Connection header received
0809 15:08:17.674616 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:17.674621 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:17.674700 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:17.674790 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:17.675914 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:17.677123 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:17.678951 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:17.679563 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:17.679704 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:17.684711 (+ 5007us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.194:50657: BlockingRecv error: recv got EOF from 127.24.183.194:50657 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 15:08:18 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:18 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:18.149344 7391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:18.140684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.195:51395)
0809 15:08:18.141202 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:18.141218 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:18.141344 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:18.141642 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:18.141645 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:18.142077 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:18.142250 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:18.142260 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:18.143280 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:18.143284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:18.144226 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:18.144239 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:18.145134 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:18.145142 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:18.145314 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:18.145928 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:18.145948 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:18.149021 (+ 3073us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":289,"threads_started":1}
W20260809 15:08:18.149510 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:18.140860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:51395 (local address 127.24.183.254:39163)
0809 15:08:18.141018 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:18.141022 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:18.141282 (+ 260us) server_negotiation.cc:407] Connection header received
0809 15:08:18.141452 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:18.141455 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:18.141531 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:18.141610 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:18.142390 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:18.143154 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:18.144379 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:18.144919 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:18.145075 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:18.149238 (+ 4163us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.195:51395: BlockingRecv error: recv got EOF from 127.24.183.195:51395 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 15:08:18 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:18 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:18.230803 7392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:18.222664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.193:41089)
0809 15:08:18.223260 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:18.223276 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:18.223392 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:18.223660 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:18.223663 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:18.224060 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:18.224238 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:18.224247 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:18.225277 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:18.225280 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:18.226184 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:18.226194 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:18.226846 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:18.226851 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:18.226981 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:18.227556 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:18.227575 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:18.230512 (+ 2937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":324,"threads_started":1}
W20260809 15:08:18.230921 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:18.222731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:41089 (local address 127.24.183.254:39163)
0809 15:08:18.222918 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:18.222924 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:18.223346 (+ 422us) server_negotiation.cc:407] Connection header received
0809 15:08:18.223483 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:18.223486 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:18.223550 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:18.223633 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:18.224382 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:18.225156 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:18.226326 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:18.226730 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:18.226827 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:18.230715 (+ 3888us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.193:41089: BlockingRecv error: recv got EOF from 127.24.183.193:41089 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 09 15:08:18 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:18 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:18.693800 7393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:18.685577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.194:42283)
0809 15:08:18.686122 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:18.686139 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:18.686255 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:18.686533 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:18.686536 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:18.686908 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:18.687157 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:18.687169 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:18.688381 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:18.688387 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:18.689280 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:18.689292 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:18.689995 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:18.690000 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:18.690172 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:18.690716 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:18.690734 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:18.693449 (+ 2715us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":264,"threads_started":1}
W20260809 15:08:18.693900 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:18.685671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:42283 (local address 127.24.183.254:39163)
0809 15:08:18.685830 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:18.685834 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:18.686207 (+ 373us) server_negotiation.cc:407] Connection header received
0809 15:08:18.686344 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:18.686348 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:18.686410 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:18.686504 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:18.687308 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:18.688240 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:18.689423 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:18.689853 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:18.689968 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:18.693670 (+ 3702us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.194:42283: BlockingRecv error: recv got EOF from 127.24.183.194:42283 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 15:08:19 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:19 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:19.158663 7394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:19.150360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.195:38435)
0809 15:08:19.150924 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:19.150941 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:19.151072 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:19.151400 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:19.151403 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:19.151808 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:19.152003 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:19.152017 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:19.152997 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:19.153000 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:19.153895 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:19.153914 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:19.154547 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:19.154550 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:19.154682 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:19.155259 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:19.155277 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:19.158302 (+ 3025us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":296,"threads_started":1}
W20260809 15:08:19.158762 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:19.150493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:38435 (local address 127.24.183.254:39163)
0809 15:08:19.150659 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:19.150664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:19.151009 (+ 345us) server_negotiation.cc:407] Connection header received
0809 15:08:19.151159 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:19.151163 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:19.151256 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:19.151378 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:19.152141 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:19.152872 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:19.154014 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:19.154427 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:19.154527 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:19.158533 (+ 4006us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.195:38435: BlockingRecv error: recv got EOF from 127.24.183.195:38435 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 15:08:19 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:19 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:19.240106 7397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:19.231869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.193:59473)
0809 15:08:19.232373 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:19.232389 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:19.232541 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:19.232853 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:19.232858 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:19.233267 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:19.233428 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:19.233437 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:19.234438 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:19.234444 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:19.235432 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:19.235445 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:19.236236 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:19.236242 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:19.236422 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:19.236928 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:19.236945 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:19.239856 (+ 2911us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":266,"threads_started":1}
W20260809 15:08:19.240311 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:19.231894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:59473 (local address 127.24.183.254:39163)
0809 15:08:19.232110 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:19.232117 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:19.232477 (+ 360us) server_negotiation.cc:407] Connection header received
0809 15:08:19.232657 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:19.232662 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:19.232730 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:19.232810 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:19.233573 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:19.234310 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:19.235574 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:19.236086 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:19.236225 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:19.240066 (+ 3841us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.193:59473: BlockingRecv error: recv got EOF from 127.24.183.193:59473 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 09 15:08:19 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:19 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:19.702997 7398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:19.694807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.194:33535)
0809 15:08:19.695339 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:19.695356 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:19.695483 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:19.695834 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:19.695837 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:19.696260 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:19.696421 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:19.696440 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:19.697441 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:19.697444 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:19.698483 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:19.698499 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:19.699421 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:19.699431 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:19.699552 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:19.700103 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:19.700121 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:19.702742 (+ 2621us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":297,"threads_started":1}
W20260809 15:08:19.703127 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:19.694830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:33535 (local address 127.24.183.254:39163)
0809 15:08:19.695027 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:19.695033 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:19.695434 (+ 401us) server_negotiation.cc:407] Connection header received
0809 15:08:19.695597 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:19.695600 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:19.695689 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:19.695803 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:19.696566 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:19.697333 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:19.698631 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:19.699299 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:19.699404 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:19.702930 (+ 3526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.194:33535: BlockingRecv error: recv got EOF from 127.24.183.194:33535 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 15:08:20 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:20 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:20.167977 7399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:20.159589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.195:38763)
0809 15:08:20.160113 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:20.160130 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:20.160248 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:20.160549 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:20.160552 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:20.160928 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:20.161086 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:20.161095 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:20.162052 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:20.162055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:20.163068 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:20.163081 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:20.163993 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:20.163997 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:20.164128 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:20.164738 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:20.164755 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:20.167623 (+ 2868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":284,"threads_started":1}
W20260809 15:08:20.168170 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:20.159649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:38763 (local address 127.24.183.254:39163)
0809 15:08:20.159823 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:20.159827 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:20.160191 (+ 364us) server_negotiation.cc:407] Connection header received
0809 15:08:20.160331 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:20.160335 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:20.160429 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:20.160508 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:20.161193 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:20.161953 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:20.163215 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:20.163852 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:20.163975 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:20.167886 (+ 3911us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.195:38763: BlockingRecv error: recv got EOF from 127.24.183.195:38763 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 15:08:20 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:20 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:20.249027 7400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:20.241108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.193:45233)
0809 15:08:20.241608 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:20.241625 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:20.241736 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:20.242000 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:20.242004 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:20.242412 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:20.242584 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:20.242592 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:20.243576 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:20.243580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:20.244602 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:20.244615 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:20.245276 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:20.245280 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:20.245397 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:20.245957 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:20.245976 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:20.248770 (+ 2794us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":243,"threads_started":1}
W20260809 15:08:20.249152 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:20.241115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:45233 (local address 127.24.183.254:39163)
0809 15:08:20.241324 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:20.241331 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:20.241704 (+ 373us) server_negotiation.cc:407] Connection header received
0809 15:08:20.241829 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:20.241833 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:20.241895 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:20.241966 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:20.242704 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:20.243454 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:20.244723 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:20.245167 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:20.245261 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:20.248970 (+ 3709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.193:45233: BlockingRecv error: recv got EOF from 127.24.183.193:45233 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 09 15:08:20 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:20 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:20.712628 7405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:20.704021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.194:41761)
0809 15:08:20.704587 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:20.704603 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:20.704731 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:20.705057 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:20.705061 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:20.705487 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:20.705664 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:20.705674 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:20.706806 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:20.706812 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:20.708042 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:20.708057 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:20.708751 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:20.708757 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:20.708922 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:20.709510 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:20.709529 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:20.712264 (+ 2735us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":264,"threads_started":1}
W20260809 15:08:20.712726 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:20.704235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:41761 (local address 127.24.183.254:39163)
0809 15:08:20.704463 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:20.704470 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:20.704675 (+ 205us) server_negotiation.cc:407] Connection header received
0809 15:08:20.704844 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:20.704849 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:20.704922 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:20.705003 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:20.705806 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:20.706672 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:20.708179 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:20.708612 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:20.708721 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:20.712478 (+ 3757us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.194:41761: BlockingRecv error: recv got EOF from 127.24.183.194:41761 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 09 15:08:21 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:21 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:21.177436 7406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:21.169028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.195:43673)
0809 15:08:21.169593 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:21.169617 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:21.169766 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:21.170183 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:21.170187 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:21.170759 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:21.170988 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:21.171005 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:21.172122 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:21.172127 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:21.173057 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:21.173068 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:21.173730 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:21.173733 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:21.173854 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:21.174430 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:21.174449 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:21.177164 (+ 2715us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":248,"threads_started":1}
W20260809 15:08:21.177587 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:21.169083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:43673 (local address 127.24.183.254:39163)
0809 15:08:21.169243 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:21.169247 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:21.169699 (+ 452us) server_negotiation.cc:407] Connection header received
0809 15:08:21.169886 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:21.169891 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:21.169998 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:21.170104 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:21.171143 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:21.171993 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:21.173188 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:21.173622 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:21.173714 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:21.177344 (+ 3630us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.195:43673: BlockingRecv error: recv got EOF from 127.24.183.195:43673 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 15:08:21 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:21 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:21.258627 7408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:21.250080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.193:41321)
0809 15:08:21.250557 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:21.250572 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:21.250706 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:21.251037 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:21.251042 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:21.251454 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:21.251619 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:21.251628 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:21.253138 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:21.253144 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:21.254107 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:21.254120 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:21.254867 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:21.254872 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:21.255007 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:21.255578 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:21.255597 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:21.258389 (+ 2792us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":260,"threads_started":1}
W20260809 15:08:21.258939 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:21.250118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:41321 (local address 127.24.183.254:39163)
0809 15:08:21.250300 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:21.250307 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:21.250657 (+ 350us) server_negotiation.cc:407] Connection header received
0809 15:08:21.250819 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:21.250824 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:21.250888 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:21.250972 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:21.251778 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:21.252963 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:21.254258 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:21.254708 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:21.254814 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:21.258633 (+ 3819us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.193:41321: BlockingRecv error: recv got EOF from 127.24.183.193:41321 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 09 15:08:21 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:21 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:21.721576 7409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:21.713687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.194:42933)
0809 15:08:21.714227 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:21.714244 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:21.714363 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:21.714625 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:21.714628 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:21.715029 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:21.715191 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:21.715201 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:21.716278 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:21.716281 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:21.717176 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:21.717187 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:21.717819 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:21.717823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:21.717948 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:21.718507 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:21.718526 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:21.721256 (+ 2730us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":287,"threads_started":1}
W20260809 15:08:21.721796 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:21.713729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:42933 (local address 127.24.183.254:39163)
0809 15:08:21.713917 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:21.713924 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:21.714322 (+ 398us) server_negotiation.cc:407] Connection header received
0809 15:08:21.714447 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:21.714450 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:21.714515 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:21.714593 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:21.715343 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:21.716170 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:21.717283 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:21.717705 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:21.717793 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:21.721548 (+ 3755us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.194:42933: BlockingRecv error: recv got EOF from 127.24.183.194:42933 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 15:08:22 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:22 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:22.191546 7411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:22.178360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.195:34277)
0809 15:08:22.178864 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:22.178881 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:22.179026 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:22.179361 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:22.179365 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:22.179811 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:22.179984 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:22.179995 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:22.181045 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:22.181049 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:22.181974 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:22.181987 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:22.182729 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:22.182735 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:22.182899 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:22.188464 (+ 5565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:22.188483 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:22.191294 (+ 2811us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":265,"threads_started":1}
W20260809 15:08:22.191789 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:22.178396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:34277 (local address 127.24.183.254:39163)
0809 15:08:22.178574 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:22.178582 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:22.178969 (+ 387us) server_negotiation.cc:407] Connection header received
0809 15:08:22.179139 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:22.179144 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:22.179221 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:22.179307 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:22.180131 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:22.180900 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:22.182118 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:22.182575 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:22.182683 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:22.191517 (+ 8834us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.195:34277: BlockingRecv error: recv got EOF from 127.24.183.195:34277 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 09 15:08:22 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:22 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:22.267228 7413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:22.259497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.193:58625)
0809 15:08:22.260083 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:22.260100 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:22.260226 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:22.260488 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:22.260491 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:22.260918 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:22.261111 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:22.261121 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:22.262028 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:22.262032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:22.262942 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:22.262953 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:22.263574 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:22.263578 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:22.263715 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:22.264272 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:22.264291 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:22.266979 (+ 2688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":266,"threads_started":1}
W20260809 15:08:22.267364 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:22.259578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:58625 (local address 127.24.183.254:39163)
0809 15:08:22.259807 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:22.259814 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:22.260181 (+ 367us) server_negotiation.cc:407] Connection header received
0809 15:08:22.260316 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:22.260320 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:22.260382 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:22.260459 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:22.261219 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:22.261918 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:22.263043 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:22.263464 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:22.263549 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:22.267176 (+ 3627us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.193:58625: BlockingRecv error: recv got EOF from 127.24.183.193:58625 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 09 15:08:22 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:22 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:22.730785 7414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:22.722526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.194:35473)
0809 15:08:22.723108 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:22.723128 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:22.723273 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:22.723641 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:22.723646 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:22.724116 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:22.724291 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:22.724301 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:22.725345 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:22.725352 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:22.726270 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:22.726282 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:22.726981 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:22.726987 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:22.727151 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:22.727656 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:22.727697 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:22.730473 (+ 2776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":252,"threads_started":1}
W20260809 15:08:22.730947 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:22.722571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:35473 (local address 127.24.183.254:39163)
0809 15:08:22.722770 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:22.722776 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:22.723207 (+ 431us) server_negotiation.cc:407] Connection header received
0809 15:08:22.723392 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:22.723397 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:22.723499 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:22.723597 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:22.724440 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:22.725208 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:22.726409 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:22.726832 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:22.726951 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:22.730718 (+ 3767us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.194:35473: BlockingRecv error: recv got EOF from 127.24.183.194:35473 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 09 15:08:23 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:23 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:23.201296 7415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:23.192479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.195:40793)
0809 15:08:23.193060 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:23.193077 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:23.193210 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:23.193553 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:23.193557 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:23.193989 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:23.194169 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:23.194178 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:23.195215 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:23.195221 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:23.196466 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:23.196484 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:23.197210 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:23.197217 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:23.197384 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:23.197958 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:23.197979 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:23.200972 (+ 2993us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":260,"threads_started":1}
W20260809 15:08:23.201457 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:23.192535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:40793 (local address 127.24.183.254:39163)
0809 15:08:23.192743 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:23.192750 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:23.193159 (+ 409us) server_negotiation.cc:407] Connection header received
0809 15:08:23.193306 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:23.193309 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:23.193391 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:23.193479 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:23.194283 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:23.195033 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:23.196605 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:23.197060 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:23.197155 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:23.201201 (+ 4046us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.195:40793: BlockingRecv error: recv got EOF from 127.24.183.195:40793 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 09 15:08:23 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:23 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:23.276580 7416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:23.268165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.193:54571)
0809 15:08:23.268696 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:23.268714 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:23.268839 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:23.269155 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:23.269159 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:23.269609 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:23.269781 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:23.269791 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:23.270881 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:23.270886 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:23.271945 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:23.271960 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:23.272761 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:23.272768 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:23.272943 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:23.273495 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:23.273514 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:23.276340 (+ 2826us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":282,"threads_started":1}
W20260809 15:08:23.276751 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:23.268309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:54571 (local address 127.24.183.254:39163)
0809 15:08:23.268462 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:23.268466 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:23.268781 (+ 315us) server_negotiation.cc:407] Connection header received
0809 15:08:23.268923 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:23.268927 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:23.268989 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:23.269081 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:23.269914 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:23.270738 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:23.272085 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:23.272577 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:23.272688 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:23.276534 (+ 3846us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.193:54571: BlockingRecv error: recv got EOF from 127.24.183.193:54571 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 09 15:08:23 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:23 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:23.740329 7417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:23.731769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.194:39483)
0809 15:08:23.732331 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:23.732348 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:23.732478 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:23.732747 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:23.732750 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:23.733167 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:23.733325 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:23.733333 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:23.734460 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:23.734464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:23.735446 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:23.735459 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:23.736212 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:23.736217 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:23.736399 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:23.736960 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:23.736979 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:23.740024 (+ 3045us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":283,"threads_started":1}
W20260809 15:08:23.740460 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:23.731787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:39483 (local address 127.24.183.254:39163)
0809 15:08:23.731961 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:23.731966 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:23.732416 (+ 450us) server_negotiation.cc:407] Connection header received
0809 15:08:23.732569 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:23.732572 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:23.732636 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:23.732716 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:23.733435 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:23.734339 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:23.735561 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:23.736047 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:23.736140 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:23.740220 (+ 4080us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.194:39483: BlockingRecv error: recv got EOF from 127.24.183.194:39483 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 15:08:24 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:24 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:24.210760 7419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:24.202336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.195:55293)
0809 15:08:24.202895 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:24.202911 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:24.203048 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:24.203399 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:24.203404 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:24.203901 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:24.204061 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:24.204071 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:24.205113 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:24.205120 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:24.206067 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:24.206080 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:24.206831 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:24.206838 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:24.206998 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:24.207566 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:24.207583 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:24.210430 (+ 2847us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":279,"threads_started":1}
W20260809 15:08:24.210906 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:24.202420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:55293 (local address 127.24.183.254:39163)
0809 15:08:24.202657 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:24.202664 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:24.202998 (+ 334us) server_negotiation.cc:407] Connection header received
0809 15:08:24.203176 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:24.203181 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:24.203262 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:24.203342 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:24.204228 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:24.204955 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:24.206217 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:24.206652 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:24.206765 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:24.210654 (+ 3889us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.195:55293: BlockingRecv error: recv got EOF from 127.24.183.195:55293 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 09 15:08:24 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:24 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:24.285888 7423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:24.277568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.193:46719)
0809 15:08:24.278134 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:24.278152 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:24.278289 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:24.278577 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:24.278580 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:24.278996 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:24.279178 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:24.279189 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:24.280178 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:24.280182 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:24.281141 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:24.281152 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:24.281777 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:24.281782 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:24.281897 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:24.282470 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:24.282489 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:24.285619 (+ 3130us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":292,"threads_started":1}
W20260809 15:08:24.286046 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:24.277614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:46719 (local address 127.24.183.254:39163)
0809 15:08:24.277792 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:24.277797 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:24.278220 (+ 423us) server_negotiation.cc:407] Connection header received
0809 15:08:24.278382 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:24.278387 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:24.278450 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:24.278536 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:24.279304 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:24.280082 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:24.281260 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:24.281665 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:24.281755 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:24.285827 (+ 4072us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.193:46719: BlockingRecv error: recv got EOF from 127.24.183.193:46719 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 15:08:24 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:24 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:24.750209 7424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:24.741515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.194:36187)
0809 15:08:24.742086 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:24.742103 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:24.742246 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:24.742591 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:24.742596 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:24.743063 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:24.743297 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:24.743310 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:24.744548 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:24.744553 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:24.745641 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:24.745656 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:24.746370 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:24.746374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:24.746508 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:24.747073 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:24.747091 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:24.749948 (+ 2857us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":320,"threads_started":1}
W20260809 15:08:24.750375 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:24.741594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:36187 (local address 127.24.183.254:39163)
0809 15:08:24.741835 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:24.741842 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:24.742188 (+ 346us) server_negotiation.cc:407] Connection header received
0809 15:08:24.742364 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:24.742369 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:24.742437 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:24.742521 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:24.743443 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:24.744390 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:24.745769 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:24.746235 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:24.746326 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:24.750150 (+ 3824us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.194:36187: BlockingRecv error: recv got EOF from 127.24.183.194:36187 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 09 15:08:25 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:25 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:25.220130 7425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:25.211975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.195:59589)
0809 15:08:25.212511 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:25.212528 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:25.212663 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:25.213032 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:25.213037 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:25.213527 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:25.213725 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:25.213740 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:25.214744 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:25.214748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:25.215747 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:25.215759 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:25.216367 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:25.216371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:25.216505 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:25.217015 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:25.217033 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:25.219874 (+ 2841us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":312,"threads_started":1}
W20260809 15:08:25.220286 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:25.212138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:59589 (local address 127.24.183.254:39163)
0809 15:08:25.212296 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:25.212300 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:25.212602 (+ 302us) server_negotiation.cc:407] Connection header received
0809 15:08:25.212805 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:25.212809 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:25.212872 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:25.212966 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:25.213889 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:25.214624 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:25.215846 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:25.216251 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:25.216346 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:25.220076 (+ 3730us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.195:59589: BlockingRecv error: recv got EOF from 127.24.183.195:59589 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 09 15:08:25 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:25 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:25.295516 7428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:25.286898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.193:40141)
0809 15:08:25.287471 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:25.287491 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:25.287618 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:25.287985 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:25.287989 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:25.288469 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:25.288659 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:25.288671 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:25.289800 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:25.289804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:25.290840 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:25.290853 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:25.291595 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:25.291603 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:25.291776 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:25.292325 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:25.292341 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:25.295212 (+ 2871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":303,"threads_started":1}
W20260809 15:08:25.295722 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:25.286949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:40141 (local address 127.24.183.254:39163)
0809 15:08:25.287123 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:25.287127 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:25.287562 (+ 435us) server_negotiation.cc:407] Connection header received
0809 15:08:25.287804 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:25.287810 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:25.287872 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:25.287958 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:25.288856 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:25.289645 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:25.290972 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:25.291424 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:25.291538 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:25.295460 (+ 3922us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.193:40141: BlockingRecv error: recv got EOF from 127.24.183.193:40141 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 09 15:08:25 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:25 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:25.759398 7431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:25.751206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.194:59903)
0809 15:08:25.751792 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:25.751809 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:25.751931 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:25.752216 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:25.752221 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:25.752743 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:25.752958 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:25.752968 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:25.754023 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:25.754027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:25.755003 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:25.755017 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:25.755655 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:25.755660 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:25.755862 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:25.756442 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:25.756460 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:25.759160 (+ 2700us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":301,"threads_started":1}
W20260809 15:08:25.759598 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:25.751255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:59903 (local address 127.24.183.254:39163)
0809 15:08:25.751440 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:25.751444 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:25.751879 (+ 435us) server_negotiation.cc:407] Connection header received
0809 15:08:25.752024 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:25.752027 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:25.752092 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:25.752174 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:25.753094 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:25.753866 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:25.755113 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:25.755519 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:25.755607 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:25.759365 (+ 3758us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.194:59903: BlockingRecv error: recv got EOF from 127.24.183.194:59903 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 09 15:08:26 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:26 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:26.229276 7432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:26.221129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.195:50189)
0809 15:08:26.221660 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:26.221677 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:26.221810 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:26.222146 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:26.222151 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:26.222644 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:26.222808 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:26.222818 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:26.223851 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:26.223855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:26.224797 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:26.224808 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:26.225468 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:26.225472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:26.225581 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:26.226175 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:26.226193 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:26.229039 (+ 2846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":287,"threads_started":1}
W20260809 15:08:26.229467 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:26.221281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:50189 (local address 127.24.183.254:39163)
0809 15:08:26.221537 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:26.221543 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:26.221778 (+ 235us) server_negotiation.cc:407] Connection header received
0809 15:08:26.221927 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:26.221933 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:26.222008 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:26.222087 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:26.222966 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:26.223736 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:26.224907 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:26.225351 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:26.225444 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:26.229224 (+ 3780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.195:50189: BlockingRecv error: recv got EOF from 127.24.183.195:50189 (error 108)
Metrics: {"server-negotiator.queue_time_us":131}
Aug 09 15:08:26 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:26 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:26.304801 7435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:26.296448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.193:47919)
0809 15:08:26.296976 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:26.296992 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:26.297132 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:26.297464 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:26.297469 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:26.297922 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:26.298121 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:26.298131 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:26.299176 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:26.299181 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:26.300116 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:26.300128 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:26.300851 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:26.300857 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:26.301034 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:26.301579 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:26.301596 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:26.304487 (+ 2891us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":266,"threads_started":1}
W20260809 15:08:26.304961 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:26.296540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:47919 (local address 127.24.183.254:39163)
0809 15:08:26.296747 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:26.296754 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:26.297079 (+ 325us) server_negotiation.cc:407] Connection header received
0809 15:08:26.297261 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:26.297265 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:26.297329 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:26.297414 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:26.298268 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:26.299040 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:26.300257 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:26.300703 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:26.300813 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:26.304700 (+ 3887us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.193:47919: BlockingRecv error: recv got EOF from 127.24.183.193:47919 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 09 15:08:26 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:26 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:26.769234 7436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:26.760410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.194:60957)
0809 15:08:26.760992 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:26.761008 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:26.761128 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:26.761429 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:26.761432 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:26.761852 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:26.762034 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:26.762043 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:26.763294 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:26.763298 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:26.764557 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:26.764576 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:26.765315 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:26.765321 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:26.765506 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:26.766045 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:26.766062 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:26.768872 (+ 2810us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":329,"threads_started":1}
W20260809 15:08:26.769347 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:26.760532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:60957 (local address 127.24.183.254:39163)
0809 15:08:26.760708 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:26.760713 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:26.761073 (+ 360us) server_negotiation.cc:407] Connection header received
0809 15:08:26.761232 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:26.761236 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:26.761299 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:26.761378 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:26.762142 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:26.763121 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:26.764718 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:26.765162 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:26.765273 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:26.769113 (+ 3840us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.194:60957: BlockingRecv error: recv got EOF from 127.24.183.194:60957 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 09 15:08:27 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:27 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:27.241677 7438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:27.230186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.195:35429)
0809 15:08:27.230764 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:27.230780 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:27.230923 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:27.231252 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:27.231257 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:27.231724 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:27.231911 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:27.231921 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:27.232996 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:27.233003 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:27.233900 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:27.233917 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:27.234625 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:27.234631 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:27.234821 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:27.238538 (+ 3717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:27.238557 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:27.241332 (+ 2775us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":340,"threads_started":1}
W20260809 15:08:27.241824 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:27.230268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:35429 (local address 127.24.183.254:39163)
0809 15:08:27.230497 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:27.230504 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:27.230872 (+ 368us) server_negotiation.cc:407] Connection header received
0809 15:08:27.231051 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:27.231056 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:27.231118 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:27.231201 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:27.232078 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:27.232858 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:27.234053 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:27.234477 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:27.234585 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:27.241592 (+ 7007us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.195:35429: BlockingRecv error: recv got EOF from 127.24.183.195:35429 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 15:08:27 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:27 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:27.314095 7439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:27.305863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.193:44199)
0809 15:08:27.306358 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:27.306374 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:27.306518 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:27.306870 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:27.306873 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:27.307268 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:27.307443 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:27.307452 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:27.308537 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:27.308541 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:27.309486 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:27.309497 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:27.310305 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:27.310310 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:27.310470 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:27.311044 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:27.311063 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:27.313851 (+ 2788us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":297,"threads_started":1}
W20260809 15:08:27.314330 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:27.305886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:44199 (local address 127.24.183.254:39163)
0809 15:08:27.306145 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:27.306154 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:27.306452 (+ 298us) server_negotiation.cc:407] Connection header received
0809 15:08:27.306636 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:27.306641 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:27.306744 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:27.306834 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:27.307577 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:27.308421 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:27.309628 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:27.310180 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:27.310290 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:27.314055 (+ 3765us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.193:44199: BlockingRecv error: recv got EOF from 127.24.183.193:44199 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 09 15:08:27 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:27 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:27.779726 7440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:27.770455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.194:37771)
0809 15:08:27.771045 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:27.771061 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:27.771206 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:27.771560 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:27.771576 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:27.772132 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:27.772301 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:27.772312 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:27.773372 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:27.773377 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:27.774490 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:27.774507 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:27.775363 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:27.775369 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:27.775599 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:27.776275 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:27.776296 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:27.779365 (+ 3069us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":353,"threads_started":1}
W20260809 15:08:27.780066 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:27.770564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:37771 (local address 127.24.183.254:39163)
0809 15:08:27.770870 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:27.770876 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:27.771161 (+ 285us) server_negotiation.cc:407] Connection header received
0809 15:08:27.771330 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:27.771335 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:27.771404 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:27.771494 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:27.772467 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:27.773210 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:27.774676 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:27.775180 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:27.775330 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:27.779649 (+ 4319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.194:37771: BlockingRecv error: recv got EOF from 127.24.183.194:37771 (error 108)
Metrics: {"server-negotiator.queue_time_us":131}
Aug 09 15:08:28 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:28 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:28.250754 7442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:28.242670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.195:38539)
0809 15:08:28.243126 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:28.243142 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:28.243277 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:28.243602 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:28.243607 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:28.244035 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:28.244190 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:28.244200 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:28.245254 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:28.245258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:28.246203 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:28.246214 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:28.246934 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:28.246940 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:28.247115 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:28.247666 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:28.247707 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:28.250427 (+ 2720us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":251,"threads_started":1}
W20260809 15:08:28.250901 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:28.242695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:38539 (local address 127.24.183.254:39163)
0809 15:08:28.242876 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:28.242882 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:28.243227 (+ 345us) server_negotiation.cc:407] Connection header received
0809 15:08:28.243388 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:28.243392 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:28.243459 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:28.243542 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:28.244347 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:28.245111 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:28.246355 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:28.246786 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:28.246900 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:28.250661 (+ 3761us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.195:38539: BlockingRecv error: recv got EOF from 127.24.183.195:38539 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 15:08:28 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:28 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:28.323140 7443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:28.315044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.193:46971)
0809 15:08:28.315553 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:28.315571 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:28.315722 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:28.316066 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:28.316070 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:28.316487 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:28.316670 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:28.316679 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:28.317700 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:28.317704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:28.318631 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:28.318642 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:28.319337 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:28.319347 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:28.319481 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:28.320044 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:28.320061 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:28.322890 (+ 2829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":235,"threads_started":1}
W20260809 15:08:28.323287 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:28.315119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:46971 (local address 127.24.183.254:39163)
0809 15:08:28.315348 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:28.315354 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:28.315641 (+ 287us) server_negotiation.cc:407] Connection header received
0809 15:08:28.315844 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:28.315850 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:28.315929 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:28.316042 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:28.316807 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:28.317583 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:28.318748 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:28.319194 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:28.319289 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:28.323064 (+ 3775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.193:46971: BlockingRecv error: recv got EOF from 127.24.183.193:46971 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 09 15:08:28 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:28 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:28.789331 7444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:28.780691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.194:49003)
0809 15:08:28.781239 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:28.781255 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:28.781369 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:28.781639 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:28.781642 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:28.782059 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:28.782238 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:28.782248 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:28.783253 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:28.783257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:28.784245 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:28.784256 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:28.784961 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:28.784966 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:28.785100 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:28.785734 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:28.785759 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:28.789061 (+ 3302us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":285,"threads_started":1}
W20260809 15:08:28.789543 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:28.780846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:49003 (local address 127.24.183.254:39163)
0809 15:08:28.781007 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:28.781011 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:28.781317 (+ 306us) server_negotiation.cc:407] Connection header received
0809 15:08:28.781460 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:28.781464 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:28.781525 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:28.781607 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:28.782347 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:28.783144 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:28.784355 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:28.784766 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:28.784859 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:28.789282 (+ 4423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.194:49003: BlockingRecv error: recv got EOF from 127.24.183.194:49003 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 15:08:29 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:29 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:29.260241 7445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:29.251764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.195:32955)
0809 15:08:29.252380 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:29.252396 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:29.252520 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:29.252791 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:29.252794 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:29.253177 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:29.253346 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:29.253356 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:29.254342 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:29.254345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:29.255310 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:29.255322 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:29.256326 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:29.256333 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:29.256524 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:29.257142 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:29.257161 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:29.259955 (+ 2794us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":356,"threads_started":1}
W20260809 15:08:29.260337 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:29.251842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:32955 (local address 127.24.183.254:39163)
0809 15:08:29.252004 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:29.252008 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:29.252486 (+ 478us) server_negotiation.cc:407] Connection header received
0809 15:08:29.252610 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:29.252613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:29.252679 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:29.252778 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:29.253474 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:29.254226 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:29.255462 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:29.256109 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:29.256231 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:29.260147 (+ 3916us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.195:32955: BlockingRecv error: recv got EOF from 127.24.183.195:32955 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 09 15:08:29 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:29 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:29.332579 7447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:29.324113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.193:60523)
0809 15:08:29.324631 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:29.324648 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:29.324784 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:29.325123 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:29.325127 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:29.325522 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:29.325709 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:29.325718 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:29.326779 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:29.326785 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:29.327755 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:29.327767 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:29.328469 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:29.328475 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:29.328649 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:29.329236 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:29.329253 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:29.332240 (+ 2987us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":258,"threads_started":1}
W20260809 15:08:29.332774 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:29.324216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:60523 (local address 127.24.183.254:39163)
0809 15:08:29.324546 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:29.324552 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:29.324720 (+ 168us) server_negotiation.cc:407] Connection header received
0809 15:08:29.324901 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:29.324906 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:29.324974 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:29.325056 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:29.325855 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:29.326639 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:29.327890 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:29.328329 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:29.328446 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:29.332462 (+ 4016us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.193:60523: BlockingRecv error: recv got EOF from 127.24.183.193:60523 (error 108)
Metrics: {"server-negotiator.queue_time_us":157}
Aug 09 15:08:29 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:29 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:29.798905 7449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:29.790402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.194:53339)
0809 15:08:29.790882 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:29.790899 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:29.791020 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:29.791318 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:29.791321 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:29.791800 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:29.792061 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:29.792073 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:29.793265 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:29.793269 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:29.794252 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:29.794267 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:29.794977 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:29.794983 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:29.795149 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:29.795773 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:29.795791 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:29.798527 (+ 2736us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":247,"threads_started":1}
W20260809 15:08:29.798975 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:29.790487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:53339 (local address 127.24.183.254:39163)
0809 15:08:29.790744 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:29.790751 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:29.790978 (+ 227us) server_negotiation.cc:407] Connection header received
0809 15:08:29.791114 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:29.791117 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:29.791187 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:29.791281 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:29.792213 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:29.793124 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:29.794397 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:29.794834 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:29.794946 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:29.798742 (+ 3796us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.194:53339: BlockingRecv error: recv got EOF from 127.24.183.194:53339 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 09 15:08:30 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:30 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.195@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:30.269711 7452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:30.261222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.195:40985)
0809 15:08:30.261727 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:30.261743 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:30.261889 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:30.262225 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:30.262230 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:30.262654 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:30.262863 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:30.262873 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:30.263946 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:30.263951 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:30.264892 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:30.264904 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:30.265623 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:30.265629 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:30.265794 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:30.266390 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:30.266409 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:30.269328 (+ 2919us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":261,"threads_started":1}
W20260809 15:08:30.269860 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:30.261336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:40985 (local address 127.24.183.254:39163)
0809 15:08:30.261586 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:30.261592 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:30.261834 (+ 242us) server_negotiation.cc:407] Connection header received
0809 15:08:30.262003 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:30.262009 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:30.262074 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:30.262159 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:30.263008 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:30.263802 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:30.265031 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:30.265485 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:30.265596 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:30.269618 (+ 4022us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.195:40985: BlockingRecv error: recv got EOF from 127.24.183.195:40985 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 09 15:08:30 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:30 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.193@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:30.342743 7455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:30.333984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.193:47253)
0809 15:08:30.334487 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:30.334505 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:30.334622 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:30.334973 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:30.334977 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:30.335448 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:30.335645 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:30.335656 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:30.336762 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:30.336768 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:30.337751 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:30.337764 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:30.338503 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:30.338508 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:30.338630 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:30.339226 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:30.339246 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:30.342403 (+ 3157us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":259,"threads_started":1}
W20260809 15:08:30.342929 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:30.334099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:47253 (local address 127.24.183.254:39163)
0809 15:08:30.334344 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:30.334351 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:30.334592 (+ 241us) server_negotiation.cc:407] Connection header received
0809 15:08:30.334724 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:30.334727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:30.334811 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:30.334898 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:30.335819 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:30.336617 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:30.337892 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:30.338362 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:30.338463 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:30.342673 (+ 4210us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.193:47253: BlockingRecv error: recv got EOF from 127.24.183.193:47253 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 09 15:08:30 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:08:30 dist-test-slave-nj21 krb5kdc[5704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.183.194@KRBTEST.COM for oryx/127.24.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:08:30.808511 7456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:30.800136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:39163 (local address 127.24.183.194:57361)
0809 15:08:30.800681 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:30.800697 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:30.800842 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:30.801197 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:30.801202 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:30.801635 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:30.801817 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:30.801827 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:30.802878 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:30.802883 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:30.803836 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:30.803848 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:30.804598 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:30.804604 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:30.804765 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:30.805316 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:30.805334 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:30.808155 (+ 2821us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.183.254:39163: Server oryx/127.24.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":269,"threads_started":1}
W20260809 15:08:30.808642 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:30.800334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:57361 (local address 127.24.183.254:39163)
0809 15:08:30.800586 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:30.800593 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:30.800781 (+ 188us) server_negotiation.cc:407] Connection header received
0809 15:08:30.800963 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:30.800968 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:30.801040 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:30.801131 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:30.801967 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:30.802730 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:30.803982 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:30.804451 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:30.804599 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:30.808390 (+ 3791us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.183.194:57361: BlockingRecv error: recv got EOF from 127.24.183.194:57361 (error 108)
Metrics: {"server-negotiator.queue_time_us":115}
I20260809 15:08:31.107435 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 6693
I20260809 15:08:31.119937 7460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:31.108542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43274 (local address 127.24.183.193:39393)
0809 15:08:31.109123 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:31.109131 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:31.109160 (+ 29us) server_negotiation.cc:407] Connection header received
0809 15:08:31.109223 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:31.109229 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:31.109492 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:31.109660 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:31.111224 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:31.112366 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:31.113615 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:31.114188 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:31.114294 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:31.116785 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:31.116794 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:31.116817 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:08:31.116853 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:31.119359 (+ 2506us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.24.183.193@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 15:08:31.119640 (+ 281us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43274: Request ticket server kudu/127.24.183.193@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":302,"threads_started":1}
W20260809 15:08:31.120042 7460 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43274: Request ticket server kudu/127.24.183.193@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 15:08:31.120319 25311 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 6693: Not authorized: Client connection negotiation failed: client connection to 127.24.183.193:39393: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.24.183.193@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 15:08:31.120514 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 6693
I20260809 15:08:31.143468 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 6958
I20260809 15:08:31.155607 7461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:31.144458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59750 (local address 127.24.183.194:45717)
0809 15:08:31.145066 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:31.145073 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:31.145107 (+ 34us) server_negotiation.cc:407] Connection header received
0809 15:08:31.145172 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:31.145179 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:31.145413 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:31.145574 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:31.146959 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:31.148127 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:31.149370 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:31.149962 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:31.150078 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:31.152587 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:31.152596 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:31.152612 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:08:31.152648 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:31.155081 (+ 2433us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.24.183.194@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 15:08:31.155358 (+ 277us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59750: Request ticket server kudu/127.24.183.194@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":283,"threads_started":1}
W20260809 15:08:31.155731 7461 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59750: Request ticket server kudu/127.24.183.194@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 15:08:31.155859 25311 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 6958: Not authorized: Client connection negotiation failed: client connection to 127.24.183.194:45717: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.24.183.194@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 15:08:31.156037 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 6958
I20260809 15:08:31.183293 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 7174
I20260809 15:08:31.195993 7462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:31.184387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55860 (local address 127.24.183.195:35015)
0809 15:08:31.184979 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:31.184987 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:31.185014 (+ 27us) server_negotiation.cc:407] Connection header received
0809 15:08:31.185080 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:31.185087 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:31.185331 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:31.185489 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:31.187039 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:31.188137 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:31.189479 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:31.190065 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:31.190176 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:31.192769 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:31.192779 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:31.192795 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:08:31.192837 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:31.195377 (+ 2540us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.24.183.195@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 15:08:31.195660 (+ 283us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55860: Request ticket server kudu/127.24.183.195@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":277,"threads_started":1}
W20260809 15:08:31.196079 25311 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 7174: Not authorized: Client connection negotiation failed: client connection to 127.24.183.195:35015: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.24.183.195@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 15:08:31.196131 7462 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55860: Request ticket server kudu/127.24.183.195@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 15:08:31.196273 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 7174
I20260809 15:08:31.226183 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 6582
I20260809 15:08:31.227598 6681 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:08:31.351297 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 6582
2026-08-09T15:08:31Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21354 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T15:08:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:08:31Z Disabled control of system clock
I20260809 15:08:31.445132 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:34891
--webserver_interface=127.24.183.254
--webserver_port=0
--builtin_ntp_servers=127.24.183.212:44259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:34891
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 15:08:31.784520 7470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:08:31.784803 7470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:08:31.784883 7470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:08:31.792285 7470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:08:31.792393 7470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:08:31.792431 7470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:08:31.792488 7470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:08:31.792532 7470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:08:31.801632 7470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:44259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:34891
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:34891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:08:31.802925 7470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:08:31.804751 7470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:08:31.816255 7479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:31.816283 7482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:31.816217 7485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:08:31.817251 7470 server_base.cc:1034] running on GCE node
I20260809 15:08:31.818032 7470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:08:31.819443 7470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:08:31.820648 7470 hybrid_clock.cc:648] HybridClock initialized: now 1786288111820589 us; error 52 us; skew 500 ppm
I20260809 15:08:31.820996 7470 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:08:31.823506 7470 webserver.cc:460] Webserver started at http://127.24.183.254:44793/ using document root <none> and password file <none>
I20260809 15:08:31.824230 7470 fs_manager.cc:359] Metadata directory not provided
I20260809 15:08:31.824347 7470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:08:31.824673 7470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:08:31.827942 7470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "79ecbc6d32374eb1967ff70f3d73044c"
format_stamp: "Formatted at 2026-08-09 15:08:31 on dist-test-slave-nj21"
server_key: "f8efdd1bf8077c051988db820dcc4803"
server_key_iv: "31b33e6ca037115a3189baa025ccfd88"
server_key_version: "encryptionkey@0"
I20260809 15:08:31.828732 7470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "79ecbc6d32374eb1967ff70f3d73044c"
format_stamp: "Formatted at 2026-08-09 15:08:31 on dist-test-slave-nj21"
server_key: "f8efdd1bf8077c051988db820dcc4803"
server_key_iv: "31b33e6ca037115a3189baa025ccfd88"
server_key_version: "encryptionkey@0"
I20260809 15:08:31.833953 7470 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 15:08:31.837445 7493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:31.838812 7470 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 15:08:31.838984 7470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "79ecbc6d32374eb1967ff70f3d73044c"
format_stamp: "Formatted at 2026-08-09 15:08:31 on dist-test-slave-nj21"
server_key: "f8efdd1bf8077c051988db820dcc4803"
server_key_iv: "31b33e6ca037115a3189baa025ccfd88"
server_key_version: "encryptionkey@0"
I20260809 15:08:31.839186 7470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:08:31.850996 7470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:08:31.851851 7470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:08:31.852116 7470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:08:31.869786 7470 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:34891
I20260809 15:08:31.869833 7585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:34891 every 8 connection(s)
I20260809 15:08:31.871098 7470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:08:31.874835 7586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:08:31.881278 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 7470
I20260809 15:08:31.881487 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:08:31.882332 25311 external_mini_cluster.cc:1428] Setting key d2c5f731d22d562f33a2f1a827e66229
I20260809 15:08:31.886029 7586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79ecbc6d32374eb1967ff70f3d73044c: Bootstrap starting.
I20260809 15:08:31.889065 7586 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79ecbc6d32374eb1967ff70f3d73044c: Neither blocks nor log segments found. Creating new log.
I20260809 15:08:31.890080 7586 log.cc:824] T 00000000000000000000000000000000 P 79ecbc6d32374eb1967ff70f3d73044c: Log is configured to *not* fsync() on all Append() calls
I20260809 15:08:31.892827 7586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79ecbc6d32374eb1967ff70f3d73044c: No bootstrap required, opened a new log
I20260809 15:08:31.895578 7591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:31.884484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33912 (local address 127.24.183.254:34891)
0809 15:08:31.885426 (+ 942us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:31.885437 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:31.885479 (+ 42us) server_negotiation.cc:407] Connection header received
0809 15:08:31.887127 (+ 1648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:31.887156 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:31.887594 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:31.887985 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:31.888591 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:31.889942 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:31.891280 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:31.891985 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:31.892150 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:31.893335 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:31.893342 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:31.893356 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:08:31.893872 (+ 516us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:31.893952 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:08:31.893973 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:31.894072 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:31.894324 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 15:08:31.894623 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":346,"threads_started":1}
I20260809 15:08:31.900990 7586 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79ecbc6d32374eb1967ff70f3d73044c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ecbc6d32374eb1967ff70f3d73044c" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 34891 } }
I20260809 15:08:31.901305 7586 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79ecbc6d32374eb1967ff70f3d73044c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:08:31.901396 7586 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79ecbc6d32374eb1967ff70f3d73044c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79ecbc6d32374eb1967ff70f3d73044c, State: Initialized, Role: FOLLOWER
I20260809 15:08:31.901957 7586 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79ecbc6d32374eb1967ff70f3d73044c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ecbc6d32374eb1967ff70f3d73044c" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 34891 } }
I20260809 15:08:31.902166 7586 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79ecbc6d32374eb1967ff70f3d73044c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:08:31.902298 7586 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79ecbc6d32374eb1967ff70f3d73044c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:08:31.902446 7586 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79ecbc6d32374eb1967ff70f3d73044c [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:08:31.905062 7586 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79ecbc6d32374eb1967ff70f3d73044c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ecbc6d32374eb1967ff70f3d73044c" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 34891 } }
I20260809 15:08:31.905551 7586 leader_election.cc:302] T 00000000000000000000000000000000 P 79ecbc6d32374eb1967ff70f3d73044c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79ecbc6d32374eb1967ff70f3d73044c; no voters:
I20260809 15:08:31.906126 7586 leader_election.cc:288] T 00000000000000000000000000000000 P 79ecbc6d32374eb1967ff70f3d73044c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:08:31.906256 7593 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79ecbc6d32374eb1967ff70f3d73044c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:08:31.906929 7593 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79ecbc6d32374eb1967ff70f3d73044c [term 1 LEADER]: Becoming Leader. State: Replica: 79ecbc6d32374eb1967ff70f3d73044c, State: Running, Role: LEADER
I20260809 15:08:31.907544 7593 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79ecbc6d32374eb1967ff70f3d73044c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ecbc6d32374eb1967ff70f3d73044c" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 34891 } }
I20260809 15:08:31.908303 7586 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79ecbc6d32374eb1967ff70f3d73044c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:08:31.910724 7594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79ecbc6d32374eb1967ff70f3d73044c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79ecbc6d32374eb1967ff70f3d73044c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ecbc6d32374eb1967ff70f3d73044c" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 34891 } } }
I20260809 15:08:31.911168 7594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79ecbc6d32374eb1967ff70f3d73044c [sys.catalog]: This master's current role is: LEADER
I20260809 15:08:31.911808 7595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79ecbc6d32374eb1967ff70f3d73044c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79ecbc6d32374eb1967ff70f3d73044c. Latest consensus state: current_term: 1 leader_uuid: "79ecbc6d32374eb1967ff70f3d73044c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ecbc6d32374eb1967ff70f3d73044c" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 34891 } } }
I20260809 15:08:31.912187 7595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79ecbc6d32374eb1967ff70f3d73044c [sys.catalog]: This master's current role is: LEADER
I20260809 15:08:31.912767 7607 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:08:31.918166 7607 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:08:31.927000 7607 catalog_manager.cc:1347] Generated new cluster ID: 0f0f44b19670440db160ae31074dcf1a
I20260809 15:08:31.927106 7607 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:08:31.949461 7607 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:08:31.950737 7607 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:08:31.959596 7607 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79ecbc6d32374eb1967ff70f3d73044c: Generated new TSK 0
I20260809 15:08:31.960420 7607 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:08:31.969758 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.193:0
--local_ip_for_outbound_sockets=127.24.183.193
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:34891
--builtin_ntp_servers=127.24.183.212:44259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 15:08:32.307276 7633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:08:32.307559 7633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:08:32.307715 7633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:08:32.315420 7633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:08:32.315523 7633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:08:32.315665 7633 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.193
I20260809 15:08:32.324616 7633 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:44259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:34891
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.193
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:08:32.325915 7633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:08:32.327749 7633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:08:32.340651 7638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:32.340579 7642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:32.340619 7639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:08:32.341387 7633 server_base.cc:1034] running on GCE node
I20260809 15:08:32.342020 7633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:08:32.343024 7633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:08:32.344246 7633 hybrid_clock.cc:648] HybridClock initialized: now 1786288112344156 us; error 67 us; skew 500 ppm
I20260809 15:08:32.344568 7633 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:08:32.350744 7633 webserver.cc:460] Webserver started at http://127.24.183.193:45789/ using document root <none> and password file <none>
I20260809 15:08:32.351475 7633 fs_manager.cc:359] Metadata directory not provided
I20260809 15:08:32.351567 7633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:08:32.351967 7633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:08:32.354873 7633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/data/instance:
uuid: "212503892b254f0f809223a712f724b7"
format_stamp: "Formatted at 2026-08-09 15:08:32 on dist-test-slave-nj21"
server_key: "1d49d9d72ff9b5e5f0f7e728f3880c43"
server_key_iv: "0e0c4927aaf24448d61c273064b45419"
server_key_version: "encryptionkey@0"
I20260809 15:08:32.355717 7633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance:
uuid: "212503892b254f0f809223a712f724b7"
format_stamp: "Formatted at 2026-08-09 15:08:32 on dist-test-slave-nj21"
server_key: "1d49d9d72ff9b5e5f0f7e728f3880c43"
server_key_iv: "0e0c4927aaf24448d61c273064b45419"
server_key_version: "encryptionkey@0"
I20260809 15:08:32.360958 7633 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260809 15:08:32.364344 7650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:32.365782 7633 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:08:32.365963 7633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/wal
uuid: "212503892b254f0f809223a712f724b7"
format_stamp: "Formatted at 2026-08-09 15:08:32 on dist-test-slave-nj21"
server_key: "1d49d9d72ff9b5e5f0f7e728f3880c43"
server_key_iv: "0e0c4927aaf24448d61c273064b45419"
server_key_version: "encryptionkey@0"
I20260809 15:08:32.366205 7633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:08:32.377388 7633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:08:32.378208 7633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:08:32.378527 7633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:08:32.379513 7633 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:08:32.381090 7633 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:08:32.381212 7633 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:32.381328 7633 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:08:32.381417 7633 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:32.415443 7633 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.193:41099
I20260809 15:08:32.415484 7838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.193:41099 every 8 connection(s)
I20260809 15:08:32.416872 7633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
I20260809 15:08:32.426944 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 7633
I20260809 15:08:32.427142 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance
I20260809 15:08:32.428114 25311 external_mini_cluster.cc:1428] Setting key 3763f3fd05d39fcfdaddcd02d9a22669
I20260809 15:08:32.429870 7842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:32.419586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:49689 (local address 127.24.183.254:34891)
0809 15:08:32.420079 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:32.420084 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:32.421353 (+ 1269us) server_negotiation.cc:407] Connection header received
0809 15:08:32.423108 (+ 1755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:32.423114 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:32.423194 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:32.423293 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:32.424440 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:32.425199 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:32.426756 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:32.427183 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:32.427304 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:32.428968 (+ 1664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:32.428975 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:32.428979 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:08:32.429003 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:32.429064 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:08:32.429082 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:32.429198 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:32.429449 (+ 251us) server_negotiation.cc:299] Negotiation successful
0809 15:08:32.429538 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":268,"threads_started":1}
I20260809 15:08:32.430949 7845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:32.420235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:34891 (local address 127.24.183.193:49689)
0809 15:08:32.421166 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:32.421209 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:32.422848 (+ 1639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:32.423524 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:32.423531 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:32.423620 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:32.424272 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:32.424290 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:32.425356 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:32.425359 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:32.426608 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:32.426620 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:32.427448 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:32.427458 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:32.427846 (+ 388us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:32.428584 (+ 738us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:08:32.428588 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:32.428642 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:08:32.428762 (+ 120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:08:32.429196 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:08:32.429206 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:08:32.429213 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 15:08:32.429406 (+ 193us) client_negotiation.cc:240] Negotiation successful
0809 15:08:32.429821 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":275,"threads_started":1}
I20260809 15:08:32.431339 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.194:0
--local_ip_for_outbound_sockets=127.24.183.194
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:34891
--builtin_ntp_servers=127.24.183.212:44259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 15:08:32.432461 7839 heartbeater.cc:347] Connected to a master server at 127.24.183.254:34891
I20260809 15:08:32.432825 7839 heartbeater.cc:464] Registering TS with master...
I20260809 15:08:32.433818 7839 heartbeater.cc:511] Master 127.24.183.254:34891 requested a full tablet report, sending...
I20260809 15:08:32.436087 7529 ts_manager.cc:194] Registered new tserver with Master: 212503892b254f0f809223a712f724b7 (127.24.183.193:41099)
I20260809 15:08:32.437593 7529 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.183.193:49689
W20260809 15:08:32.764981 7847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:08:32.765250 7847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:08:32.765360 7847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:08:32.773555 7847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:08:32.773681 7847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:08:32.773826 7847 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.194
I20260809 15:08:32.783244 7847 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:44259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:34891
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.194
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:08:32.784637 7847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:08:32.786417 7847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:08:32.799033 7856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:32.799084 7852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:32.799110 7853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:08:32.800235 7847 server_base.cc:1034] running on GCE node
I20260809 15:08:32.800802 7847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:08:32.801667 7847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:08:32.802855 7847 hybrid_clock.cc:648] HybridClock initialized: now 1786288112802808 us; error 52 us; skew 500 ppm
I20260809 15:08:32.803164 7847 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:08:32.805335 7847 webserver.cc:460] Webserver started at http://127.24.183.194:33943/ using document root <none> and password file <none>
I20260809 15:08:32.806005 7847 fs_manager.cc:359] Metadata directory not provided
I20260809 15:08:32.806099 7847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:08:32.806411 7847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:08:32.809280 7847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/data/instance:
uuid: "49cf8f8d87d34443a76a6d20286afcab"
format_stamp: "Formatted at 2026-08-09 15:08:32 on dist-test-slave-nj21"
server_key: "b5651d775b8dfba292970b9edf4001dd"
server_key_iv: "62885f269fc09b6c869c2e1534854e06"
server_key_version: "encryptionkey@0"
I20260809 15:08:32.810098 7847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance:
uuid: "49cf8f8d87d34443a76a6d20286afcab"
format_stamp: "Formatted at 2026-08-09 15:08:32 on dist-test-slave-nj21"
server_key: "b5651d775b8dfba292970b9edf4001dd"
server_key_iv: "62885f269fc09b6c869c2e1534854e06"
server_key_version: "encryptionkey@0"
I20260809 15:08:32.815052 7847 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 15:08:32.818446 7867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:32.819769 7847 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 15:08:32.819936 7847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/wal
uuid: "49cf8f8d87d34443a76a6d20286afcab"
format_stamp: "Formatted at 2026-08-09 15:08:32 on dist-test-slave-nj21"
server_key: "b5651d775b8dfba292970b9edf4001dd"
server_key_iv: "62885f269fc09b6c869c2e1534854e06"
server_key_version: "encryptionkey@0"
I20260809 15:08:32.820132 7847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:08:32.846771 7847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:08:32.847630 7847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:08:32.847934 7847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:08:32.848958 7847 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:08:32.850410 7847 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:08:32.850500 7847 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:32.850605 7847 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:08:32.850674 7847 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:32.882731 7847 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.194:35397
I20260809 15:08:32.882787 8054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.194:35397 every 8 connection(s)
I20260809 15:08:32.884038 7847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
I20260809 15:08:32.888999 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 7847
I20260809 15:08:32.889143 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance
I20260809 15:08:32.889591 25311 external_mini_cluster.cc:1428] Setting key 9f4f375d71a7d188b8bd21b4f56a2bf7
I20260809 15:08:32.892773 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.195:0
--local_ip_for_outbound_sockets=127.24.183.195
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:34891
--builtin_ntp_servers=127.24.183.212:44259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 15:08:32.896875 7842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:32.886532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:46645 (local address 127.24.183.254:34891)
0809 15:08:32.886747 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:32.886754 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:32.888182 (+ 1428us) server_negotiation.cc:407] Connection header received
0809 15:08:32.890021 (+ 1839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:32.890027 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:32.890117 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:32.890212 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:32.891577 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:32.892636 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:32.894211 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:32.894773 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:32.894884 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:32.896172 (+ 1288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:32.896178 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:32.896182 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:08:32.896203 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:32.896255 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:08:32.896270 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:32.896357 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:32.896516 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 15:08:32.896609 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 15:08:32.897812 8061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:32.887075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:34891 (local address 127.24.183.194:46645)
0809 15:08:32.888025 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:32.888067 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:32.889761 (+ 1694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:32.890500 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:32.890508 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:32.890615 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:32.891415 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:32.891444 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:32.892797 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:32.892801 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:32.894064 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:32.894077 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:32.894987 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:32.894991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:32.895189 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:32.895871 (+ 682us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:08:32.895874 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:32.895913 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:08:32.895993 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:08:32.896351 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:08:32.896357 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:08:32.896363 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 15:08:32.896494 (+ 131us) client_negotiation.cc:240] Negotiation successful
0809 15:08:32.896771 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":268,"threads_started":1}
I20260809 15:08:32.899084 8057 heartbeater.cc:347] Connected to a master server at 127.24.183.254:34891
I20260809 15:08:32.899413 8057 heartbeater.cc:464] Registering TS with master...
I20260809 15:08:32.900302 8057 heartbeater.cc:511] Master 127.24.183.254:34891 requested a full tablet report, sending...
I20260809 15:08:32.901744 7529 ts_manager.cc:194] Registered new tserver with Master: 49cf8f8d87d34443a76a6d20286afcab (127.24.183.194:35397)
I20260809 15:08:32.902601 7529 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.183.194:46645
W20260809 15:08:33.230154 8062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:08:33.230417 8062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:08:33.230544 8062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:08:33.237612 8062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:08:33.237700 8062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:08:33.237846 8062 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.195
I20260809 15:08:33.246925 8062 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:44259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:34891
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.195
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:08:33.248260 8062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:08:33.250336 8062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:08:33.265205 8074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:33.265579 8080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:33.265232 8073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:08:33.266414 8062 server_base.cc:1034] running on GCE node
I20260809 15:08:33.267117 8062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:08:33.268273 8062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:08:33.269522 8062 hybrid_clock.cc:648] HybridClock initialized: now 1786288113269458 us; error 49 us; skew 500 ppm
I20260809 15:08:33.269901 8062 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:08:33.272670 8062 webserver.cc:460] Webserver started at http://127.24.183.195:45041/ using document root <none> and password file <none>
I20260809 15:08:33.273567 8062 fs_manager.cc:359] Metadata directory not provided
I20260809 15:08:33.273690 8062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:08:33.274078 8062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:08:33.278085 8062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/data/instance:
uuid: "82ddfb3689ed41f197afa90e6e243c21"
format_stamp: "Formatted at 2026-08-09 15:08:33 on dist-test-slave-nj21"
server_key: "76faf68171ef26954deec44f14161e60"
server_key_iv: "27c4171edaf55e841f6e2b5cd0ede1bf"
server_key_version: "encryptionkey@0"
I20260809 15:08:33.279106 8062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance:
uuid: "82ddfb3689ed41f197afa90e6e243c21"
format_stamp: "Formatted at 2026-08-09 15:08:33 on dist-test-slave-nj21"
server_key: "76faf68171ef26954deec44f14161e60"
server_key_iv: "27c4171edaf55e841f6e2b5cd0ede1bf"
server_key_version: "encryptionkey@0"
I20260809 15:08:33.285753 8062 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260809 15:08:33.290171 8088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:33.291865 8062 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:08:33.292050 8062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/wal
uuid: "82ddfb3689ed41f197afa90e6e243c21"
format_stamp: "Formatted at 2026-08-09 15:08:33 on dist-test-slave-nj21"
server_key: "76faf68171ef26954deec44f14161e60"
server_key_iv: "27c4171edaf55e841f6e2b5cd0ede1bf"
server_key_version: "encryptionkey@0"
I20260809 15:08:33.292303 8062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:08:33.332896 8062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:08:33.333801 8062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:08:33.334105 8062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:08:33.335135 8062 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:08:33.336691 8062 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:08:33.336817 8062 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:33.336941 8062 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:08:33.337018 8062 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:33.376678 8062 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.195:36391
I20260809 15:08:33.376737 8287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.195:36391 every 8 connection(s)
I20260809 15:08:33.378015 8062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
I20260809 15:08:33.380738 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 8062
I20260809 15:08:33.380908 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance
I20260809 15:08:33.381464 25311 external_mini_cluster.cc:1428] Setting key 5cd0dcab5bc50cbf67c4ee653e3c344a
I20260809 15:08:33.392853 7842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:33.380666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:47843 (local address 127.24.183.254:34891)
0809 15:08:33.380883 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:33.380889 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:33.382775 (+ 1886us) server_negotiation.cc:407] Connection header received
0809 15:08:33.385357 (+ 2582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:33.385362 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:33.385434 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:33.385513 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:33.386890 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:33.387620 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:33.389990 (+ 2370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:33.390406 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:33.390501 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:33.392127 (+ 1626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:33.392132 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:33.392135 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:08:33.392155 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:33.392190 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:08:33.392202 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:33.392270 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:33.392515 (+ 245us) server_negotiation.cc:299] Negotiation successful
0809 15:08:33.392601 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 15:08:33.394439 8291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:33.381377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:34891 (local address 127.24.183.195:47843)
0809 15:08:33.382598 (+ 1221us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:33.382646 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:33.385069 (+ 2423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:33.385787 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:33.385799 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:33.385917 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:33.386714 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:33.386739 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:33.387794 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:33.387800 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:33.389828 (+ 2028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:33.389845 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:33.390634 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:33.390642 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:33.390955 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:33.391740 (+ 785us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:08:33.391744 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:33.391819 (+ 75us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:08:33.391933 (+ 114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:08:33.392289 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:08:33.392299 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:08:33.392307 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 15:08:33.392486 (+ 179us) client_negotiation.cc:240] Negotiation successful
0809 15:08:33.392926 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":370,"threads_started":1}
I20260809 15:08:33.395982 8289 heartbeater.cc:347] Connected to a master server at 127.24.183.254:34891
I20260809 15:08:33.396332 8289 heartbeater.cc:464] Registering TS with master...
I20260809 15:08:33.397190 8289 heartbeater.cc:511] Master 127.24.183.254:34891 requested a full tablet report, sending...
I20260809 15:08:33.398654 7523 ts_manager.cc:194] Registered new tserver with Master: 82ddfb3689ed41f197afa90e6e243c21 (127.24.183.195:36391)
I20260809 15:08:33.399536 7523 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.183.195:47843
I20260809 15:08:33.409943 25311 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 15:08:33.422751 7842 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:33.420632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33920 (local address 127.24.183.254:34891)
0809 15:08:33.420819 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:33.420826 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:33.421497 (+ 671us) server_negotiation.cc:407] Connection header received
0809 15:08:33.421669 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:33.421674 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:33.421771 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:33.421865 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:33.422499 (+ 634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33920: BlockingRecv error: recv got EOF from 127.0.0.1:33920 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 15:08:33.423373 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 7633
I20260809 15:08:33.441020 7839 heartbeater.cc:503] Master 127.24.183.254:34891 was elected leader, sending a full tablet report...
I20260809 15:08:33.443794 8300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:33.424439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59824 (local address 127.24.183.193:41099)
0809 15:08:33.436129 (+ 11690us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:33.436136 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:33.436164 (+ 28us) server_negotiation.cc:407] Connection header received
0809 15:08:33.436220 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:33.436225 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:33.436461 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:33.436583 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:33.437111 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:33.438121 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:33.440062 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:33.440625 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:33.440745 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:33.441999 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:33.442011 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:33.442029 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:08:33.442809 (+ 780us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:33.442873 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:08:33.442901 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:33.443019 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:33.443187 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 15:08:33.443340 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11510,"thread_start_us":11392,"threads_started":1}
I20260809 15:08:33.445148 7833 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:08:33.529306 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 7633
I20260809 15:08:33.553539 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 7847
I20260809 15:08:33.561182 8303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:33.554500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50672 (local address 127.24.183.194:35397)
0809 15:08:33.555086 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:33.555094 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:33.555123 (+ 29us) server_negotiation.cc:407] Connection header received
0809 15:08:33.555196 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:33.555203 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:33.555434 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:33.555575 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:33.556019 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:33.557045 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:33.558283 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:33.558848 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:33.558942 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:33.559910 (+ 968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:33.559918 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:33.559931 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:08:33.560485 (+ 554us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:33.560534 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:08:33.560554 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:33.560645 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:33.560750 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 15:08:33.560877 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":280,"threads_started":1}
I20260809 15:08:33.562420 8046 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:08:33.643575 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 7847
I20260809 15:08:33.667042 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 8062
I20260809 15:08:33.675287 8307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:33.668098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37222 (local address 127.24.183.195:36391)
0809 15:08:33.668790 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:33.668821 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:33.668853 (+ 32us) server_negotiation.cc:407] Connection header received
0809 15:08:33.668916 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:33.668922 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:33.669155 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:33.669278 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:33.669679 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:33.670721 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:33.672173 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:33.672722 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:33.672826 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:33.673767 (+ 941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:33.673776 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:33.673790 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:08:33.674382 (+ 592us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:33.674428 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:08:33.674449 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:33.674540 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:33.674668 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 15:08:33.674835 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":378,"threads_started":1}
I20260809 15:08:33.676772 8280 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:08:33.761482 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 8062
I20260809 15:08:33.783150 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 7470
I20260809 15:08:33.784389 7580 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:08:33.891506 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 7470
2026-08-09T15:08:33Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2532 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T15:08:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:08:33Z Disabled control of system clock
I20260809 15:08:33.976511 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:42635
--webserver_interface=127.24.183.254
--webserver_port=0
--builtin_ntp_servers=127.24.183.212:35293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:42635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 15:08:34.309020 8318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:08:34.309293 8318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:08:34.309384 8318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:08:34.316370 8318 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:08:34.316465 8318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:08:34.316509 8318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:08:34.316566 8318 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:08:34.316608 8318 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:08:34.325394 8318 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:35293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:42635
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:42635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:08:34.326567 8318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:08:34.328431 8318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:08:34.339790 8325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:34.339854 8328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:34.339830 8326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:08:34.340998 8318 server_base.cc:1034] running on GCE node
I20260809 15:08:34.341746 8318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:08:34.343097 8318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:08:34.344322 8318 hybrid_clock.cc:648] HybridClock initialized: now 1786288114344266 us; error 43 us; skew 500 ppm
I20260809 15:08:34.344663 8318 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:08:34.346977 8318 webserver.cc:460] Webserver started at http://127.24.183.254:39423/ using document root <none> and password file <none>
I20260809 15:08:34.347656 8318 fs_manager.cc:359] Metadata directory not provided
I20260809 15:08:34.347793 8318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:08:34.348153 8318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:08:34.350961 8318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "cc98cb17634b4e1c82e1bb33d114999b"
format_stamp: "Formatted at 2026-08-09 15:08:34 on dist-test-slave-nj21"
server_key: "cb6c21df2bf61fe688404e73260fe0f5"
server_key_iv: "37aab2d9b523e0fea3b5f9a9b85510b5"
server_key_version: "encryptionkey@0"
I20260809 15:08:34.351764 8318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "cc98cb17634b4e1c82e1bb33d114999b"
format_stamp: "Formatted at 2026-08-09 15:08:34 on dist-test-slave-nj21"
server_key: "cb6c21df2bf61fe688404e73260fe0f5"
server_key_iv: "37aab2d9b523e0fea3b5f9a9b85510b5"
server_key_version: "encryptionkey@0"
I20260809 15:08:34.356600 8318 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260809 15:08:34.359884 8337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:34.361133 8318 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:08:34.361291 8318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "cc98cb17634b4e1c82e1bb33d114999b"
format_stamp: "Formatted at 2026-08-09 15:08:34 on dist-test-slave-nj21"
server_key: "cb6c21df2bf61fe688404e73260fe0f5"
server_key_iv: "37aab2d9b523e0fea3b5f9a9b85510b5"
server_key_version: "encryptionkey@0"
I20260809 15:08:34.361490 8318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:08:34.367087 8318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:08:34.367833 8318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:08:34.368095 8318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:08:34.385921 8318 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:42635
I20260809 15:08:34.385975 8421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:42635 every 8 connection(s)
I20260809 15:08:34.387224 8318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:08:34.388933 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 8318
I20260809 15:08:34.389132 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:08:34.389776 25311 external_mini_cluster.cc:1428] Setting key e1460bf501dc35cca26a64590c25cadf
I20260809 15:08:34.391498 8422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:08:34.397861 8426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:34.391794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60176 (local address 127.24.183.254:42635)
0809 15:08:34.392756 (+ 962us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:34.392767 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:34.392809 (+ 42us) server_negotiation.cc:407] Connection header received
0809 15:08:34.394428 (+ 1619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:34.394450 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:34.394845 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:34.395156 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:34.396304 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:34.396310 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:34.396322 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:08:34.396336 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:34.396419 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:08:34.396435 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:34.396612 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:34.396704 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 15:08:34.396942 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":350,"threads_started":1}
I20260809 15:08:34.404147 8422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc98cb17634b4e1c82e1bb33d114999b: Bootstrap starting.
I20260809 15:08:34.407119 8422 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc98cb17634b4e1c82e1bb33d114999b: Neither blocks nor log segments found. Creating new log.
I20260809 15:08:34.408138 8422 log.cc:824] T 00000000000000000000000000000000 P cc98cb17634b4e1c82e1bb33d114999b: Log is configured to *not* fsync() on all Append() calls
I20260809 15:08:34.410763 8422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc98cb17634b4e1c82e1bb33d114999b: No bootstrap required, opened a new log
I20260809 15:08:34.417742 8422 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc98cb17634b4e1c82e1bb33d114999b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc98cb17634b4e1c82e1bb33d114999b" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 42635 } }
I20260809 15:08:34.418046 8422 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc98cb17634b4e1c82e1bb33d114999b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:08:34.418135 8422 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc98cb17634b4e1c82e1bb33d114999b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc98cb17634b4e1c82e1bb33d114999b, State: Initialized, Role: FOLLOWER
I20260809 15:08:34.418674 8422 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc98cb17634b4e1c82e1bb33d114999b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc98cb17634b4e1c82e1bb33d114999b" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 42635 } }
I20260809 15:08:34.418888 8422 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc98cb17634b4e1c82e1bb33d114999b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:08:34.419010 8422 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc98cb17634b4e1c82e1bb33d114999b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:08:34.419157 8422 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc98cb17634b4e1c82e1bb33d114999b [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:08:34.421265 8422 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc98cb17634b4e1c82e1bb33d114999b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc98cb17634b4e1c82e1bb33d114999b" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 42635 } }
I20260809 15:08:34.421715 8422 leader_election.cc:302] T 00000000000000000000000000000000 P cc98cb17634b4e1c82e1bb33d114999b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc98cb17634b4e1c82e1bb33d114999b; no voters:
I20260809 15:08:34.422312 8422 leader_election.cc:288] T 00000000000000000000000000000000 P cc98cb17634b4e1c82e1bb33d114999b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:08:34.422461 8430 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc98cb17634b4e1c82e1bb33d114999b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:08:34.423450 8430 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc98cb17634b4e1c82e1bb33d114999b [term 1 LEADER]: Becoming Leader. State: Replica: cc98cb17634b4e1c82e1bb33d114999b, State: Running, Role: LEADER
I20260809 15:08:34.424103 8430 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc98cb17634b4e1c82e1bb33d114999b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc98cb17634b4e1c82e1bb33d114999b" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 42635 } }
I20260809 15:08:34.424347 8422 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc98cb17634b4e1c82e1bb33d114999b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:08:34.426846 8432 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc98cb17634b4e1c82e1bb33d114999b [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc98cb17634b4e1c82e1bb33d114999b. Latest consensus state: current_term: 1 leader_uuid: "cc98cb17634b4e1c82e1bb33d114999b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc98cb17634b4e1c82e1bb33d114999b" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 42635 } } }
I20260809 15:08:34.427271 8432 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc98cb17634b4e1c82e1bb33d114999b [sys.catalog]: This master's current role is: LEADER
I20260809 15:08:34.428125 8431 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc98cb17634b4e1c82e1bb33d114999b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc98cb17634b4e1c82e1bb33d114999b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc98cb17634b4e1c82e1bb33d114999b" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 42635 } } }
I20260809 15:08:34.428645 8431 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc98cb17634b4e1c82e1bb33d114999b [sys.catalog]: This master's current role is: LEADER
I20260809 15:08:34.430485 8447 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:08:34.435796 8447 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:08:34.445130 8447 catalog_manager.cc:1347] Generated new cluster ID: b8e8a1975d574493975b1cee3c4640ed
I20260809 15:08:34.445225 8447 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:08:34.471700 8447 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:08:34.472090 8447 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:08:34.479501 8447 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc98cb17634b4e1c82e1bb33d114999b: Generated new TSK 0
I20260809 15:08:34.480299 8447 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:08:34.492169 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.193:0
--local_ip_for_outbound_sockets=127.24.183.193
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:42635
--builtin_ntp_servers=127.24.183.212:35293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 15:08:34.841003 8464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:08:34.841253 8464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:08:34.841346 8464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:08:34.848388 8464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:08:34.848505 8464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:08:34.848680 8464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.193
I20260809 15:08:34.857553 8464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:35293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:42635
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.193
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:08:34.858789 8464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:08:34.860544 8464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:08:34.873168 8475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:34.873260 8478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:34.873142 8476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:08:34.874076 8464 server_base.cc:1034] running on GCE node
I20260809 15:08:34.874647 8464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:08:34.875535 8464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:08:34.876749 8464 hybrid_clock.cc:648] HybridClock initialized: now 1786288114876683 us; error 32 us; skew 500 ppm
I20260809 15:08:34.877054 8464 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:08:34.879209 8464 webserver.cc:460] Webserver started at http://127.24.183.193:44911/ using document root <none> and password file <none>
I20260809 15:08:34.879953 8464 fs_manager.cc:359] Metadata directory not provided
I20260809 15:08:34.880074 8464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:08:34.880456 8464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:08:34.883327 8464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/data/instance:
uuid: "81a7d404d9b34ffb9a748ff83ed3853a"
format_stamp: "Formatted at 2026-08-09 15:08:34 on dist-test-slave-nj21"
server_key: "4562626cf93aca2c4f8c3929066efc57"
server_key_iv: "8351a2b77271497392a21cf6f57513ec"
server_key_version: "encryptionkey@0"
I20260809 15:08:34.884104 8464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance:
uuid: "81a7d404d9b34ffb9a748ff83ed3853a"
format_stamp: "Formatted at 2026-08-09 15:08:34 on dist-test-slave-nj21"
server_key: "4562626cf93aca2c4f8c3929066efc57"
server_key_iv: "8351a2b77271497392a21cf6f57513ec"
server_key_version: "encryptionkey@0"
I20260809 15:08:34.888962 8464 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 15:08:34.892266 8489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:34.893441 8464 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.002s
I20260809 15:08:34.893582 8464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/wal
uuid: "81a7d404d9b34ffb9a748ff83ed3853a"
format_stamp: "Formatted at 2026-08-09 15:08:34 on dist-test-slave-nj21"
server_key: "4562626cf93aca2c4f8c3929066efc57"
server_key_iv: "8351a2b77271497392a21cf6f57513ec"
server_key_version: "encryptionkey@0"
I20260809 15:08:34.893806 8464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:08:34.899222 8464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:08:34.900002 8464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:08:34.900279 8464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:08:34.901188 8464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:08:34.902499 8464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:08:34.902594 8464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:34.902697 8464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:08:34.902781 8464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:34.933426 8464 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.193:41167
I20260809 15:08:34.933470 8669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.193:41167 every 8 connection(s)
I20260809 15:08:34.934722 8464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
I20260809 15:08:34.941124 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 8464
I20260809 15:08:34.941318 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance
I20260809 15:08:34.941996 25311 external_mini_cluster.cc:1428] Setting key 6f484846d310e00665a613032c44d67d
I20260809 15:08:34.942801 8674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:34.937278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:38361 (local address 127.24.183.254:42635)
0809 15:08:34.937751 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:34.937756 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:34.938793 (+ 1037us) server_negotiation.cc:407] Connection header received
0809 15:08:34.940530 (+ 1737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:34.940536 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:34.940608 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:34.940728 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:34.942014 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:34.942020 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:34.942025 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:08:34.942043 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:34.942090 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:08:34.942105 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:34.942235 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:34.942394 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 15:08:34.942464 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":225,"threads_started":1}
I20260809 15:08:34.943765 8675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:34.937722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:42635 (local address 127.24.183.193:38361)
0809 15:08:34.938608 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:34.938647 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:34.940264 (+ 1617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:34.940960 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:34.940968 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:34.941055 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:34.941713 (+ 658us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:08:34.941716 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:34.941756 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:08:34.941841 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:08:34.942189 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:08:34.942200 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:08:34.942208 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 15:08:34.942466 (+ 258us) client_negotiation.cc:240] Negotiation successful
0809 15:08:34.942715 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":276,"threads_started":1}
I20260809 15:08:34.944758 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.194:0
--local_ip_for_outbound_sockets=127.24.183.194
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:42635
--builtin_ntp_servers=127.24.183.212:35293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 15:08:34.945221 8671 heartbeater.cc:347] Connected to a master server at 127.24.183.254:42635
I20260809 15:08:34.945521 8671 heartbeater.cc:464] Registering TS with master...
I20260809 15:08:34.946192 8671 heartbeater.cc:511] Master 127.24.183.254:42635 requested a full tablet report, sending...
I20260809 15:08:34.948397 8371 ts_manager.cc:194] Registered new tserver with Master: 81a7d404d9b34ffb9a748ff83ed3853a (127.24.183.193:41167)
W20260809 15:08:35.288290 8676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:08:35.288558 8676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:08:35.288652 8676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:08:35.295944 8676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:08:35.296039 8676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:08:35.296156 8676 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.194
I20260809 15:08:35.305207 8676 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:35293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:42635
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.194
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:08:35.306473 8676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:08:35.308285 8676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:08:35.321107 8685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:35.321301 8689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:35.321209 8684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:08:35.322265 8676 server_base.cc:1034] running on GCE node
I20260809 15:08:35.322854 8676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:08:35.323766 8676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:08:35.324965 8676 hybrid_clock.cc:648] HybridClock initialized: now 1786288115324922 us; error 32 us; skew 500 ppm
I20260809 15:08:35.325256 8676 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:08:35.327387 8676 webserver.cc:460] Webserver started at http://127.24.183.194:40545/ using document root <none> and password file <none>
I20260809 15:08:35.328100 8676 fs_manager.cc:359] Metadata directory not provided
I20260809 15:08:35.328204 8676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:08:35.328519 8676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:08:35.331271 8676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/data/instance:
uuid: "8b6f92ff01634cf2919435c5ca7ee396"
format_stamp: "Formatted at 2026-08-09 15:08:35 on dist-test-slave-nj21"
server_key: "352c127341a52a6de0b2ee024fa41e1c"
server_key_iv: "cef36ff930d31d3f7435a3da8ba7fa91"
server_key_version: "encryptionkey@0"
I20260809 15:08:35.332072 8676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance:
uuid: "8b6f92ff01634cf2919435c5ca7ee396"
format_stamp: "Formatted at 2026-08-09 15:08:35 on dist-test-slave-nj21"
server_key: "352c127341a52a6de0b2ee024fa41e1c"
server_key_iv: "cef36ff930d31d3f7435a3da8ba7fa91"
server_key_version: "encryptionkey@0"
I20260809 15:08:35.337033 8676 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260809 15:08:35.340304 8699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:35.341549 8676 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:08:35.341694 8676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/wal
uuid: "8b6f92ff01634cf2919435c5ca7ee396"
format_stamp: "Formatted at 2026-08-09 15:08:35 on dist-test-slave-nj21"
server_key: "352c127341a52a6de0b2ee024fa41e1c"
server_key_iv: "cef36ff930d31d3f7435a3da8ba7fa91"
server_key_version: "encryptionkey@0"
I20260809 15:08:35.341915 8676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:08:35.347321 8676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:08:35.348165 8676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:08:35.348407 8676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:08:35.349253 8676 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:08:35.350576 8676 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:08:35.350673 8676 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:35.350775 8676 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:08:35.350844 8676 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:35.382211 8676 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.194:38771
I20260809 15:08:35.382287 8877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.194:38771 every 8 connection(s)
I20260809 15:08:35.383539 8676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
I20260809 15:08:35.392635 8674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:35.386142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:60831 (local address 127.24.183.254:42635)
0809 15:08:35.386346 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:35.386352 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:35.387852 (+ 1500us) server_negotiation.cc:407] Connection header received
0809 15:08:35.390297 (+ 2445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:35.390302 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:35.390367 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:35.390462 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:35.391869 (+ 1407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:35.391874 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:35.391877 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:08:35.391894 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:35.391929 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:08:35.391941 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:35.392041 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:35.392263 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 15:08:35.392323 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 15:08:35.392828 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 8676
I20260809 15:08:35.393013 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance
I20260809 15:08:35.393471 25311 external_mini_cluster.cc:1428] Setting key 1f0638596b8f0047ca98c428658e3436
I20260809 15:08:35.393635 8883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:35.386634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:42635 (local address 127.24.183.194:60831)
0809 15:08:35.387703 (+ 1069us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:35.387748 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:35.389984 (+ 2236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:35.390710 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:35.390722 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:35.390835 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:35.391517 (+ 682us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:08:35.391521 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:35.391570 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:08:35.391695 (+ 125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:08:35.392032 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:08:35.392041 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:08:35.392049 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 15:08:35.392301 (+ 252us) client_negotiation.cc:240] Negotiation successful
0809 15:08:35.392550 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":296,"threads_started":1}
I20260809 15:08:35.395045 8878 heartbeater.cc:347] Connected to a master server at 127.24.183.254:42635
I20260809 15:08:35.395372 8878 heartbeater.cc:464] Registering TS with master...
I20260809 15:08:35.396082 8878 heartbeater.cc:511] Master 127.24.183.254:42635 requested a full tablet report, sending...
I20260809 15:08:35.396821 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.195:0
--local_ip_for_outbound_sockets=127.24.183.195
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:42635
--builtin_ntp_servers=127.24.183.212:35293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 15:08:35.397508 8370 ts_manager.cc:194] Registered new tserver with Master: 8b6f92ff01634cf2919435c5ca7ee396 (127.24.183.194:38771)
W20260809 15:08:35.734175 8884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:08:35.734453 8884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:08:35.734592 8884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:08:35.741982 8884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:08:35.742080 8884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:08:35.742224 8884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.195
I20260809 15:08:35.751202 8884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:35293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:42635
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.195
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:08:35.752492 8884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:08:35.754278 8884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:08:35.767374 8891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:35.767333 8894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:35.767513 8898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:08:35.768362 8884 server_base.cc:1034] running on GCE node
I20260809 15:08:35.768987 8884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:08:35.769891 8884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:08:35.771096 8884 hybrid_clock.cc:648] HybridClock initialized: now 1786288115771041 us; error 44 us; skew 500 ppm
I20260809 15:08:35.771395 8884 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:08:35.773541 8884 webserver.cc:460] Webserver started at http://127.24.183.195:38711/ using document root <none> and password file <none>
I20260809 15:08:35.774191 8884 fs_manager.cc:359] Metadata directory not provided
I20260809 15:08:35.774295 8884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:08:35.774606 8884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:08:35.777419 8884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/data/instance:
uuid: "d59fe92dce8b4e17b3685df44fa33974"
format_stamp: "Formatted at 2026-08-09 15:08:35 on dist-test-slave-nj21"
server_key: "0249f466fbf2b704875b2c85cb1234c6"
server_key_iv: "ea545185d74bc9609897e229410d0aed"
server_key_version: "encryptionkey@0"
I20260809 15:08:35.778179 8884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance:
uuid: "d59fe92dce8b4e17b3685df44fa33974"
format_stamp: "Formatted at 2026-08-09 15:08:35 on dist-test-slave-nj21"
server_key: "0249f466fbf2b704875b2c85cb1234c6"
server_key_iv: "ea545185d74bc9609897e229410d0aed"
server_key_version: "encryptionkey@0"
I20260809 15:08:35.782987 8884 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260809 15:08:35.786420 8908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:35.787802 8884 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:08:35.787958 8884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/wal
uuid: "d59fe92dce8b4e17b3685df44fa33974"
format_stamp: "Formatted at 2026-08-09 15:08:35 on dist-test-slave-nj21"
server_key: "0249f466fbf2b704875b2c85cb1234c6"
server_key_iv: "ea545185d74bc9609897e229410d0aed"
server_key_version: "encryptionkey@0"
I20260809 15:08:35.788152 8884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:08:35.793630 8884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:08:35.794378 8884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:08:35.794627 8884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:08:35.795511 8884 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:08:35.796908 8884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:08:35.797012 8884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:35.797117 8884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:08:35.797200 8884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:35.827922 8884 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.195:41083
I20260809 15:08:35.827988 9102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.195:41083 every 8 connection(s)
I20260809 15:08:35.829233 8884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
I20260809 15:08:35.833701 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 8884
I20260809 15:08:35.833904 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance
I20260809 15:08:35.834446 25311 external_mini_cluster.cc:1428] Setting key 2863de4cd1d89d2ead7106afe1381eec
I20260809 15:08:35.837788 8674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:35.831845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:55657 (local address 127.24.183.254:42635)
0809 15:08:35.832037 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:35.832042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:35.833453 (+ 1411us) server_negotiation.cc:407] Connection header received
0809 15:08:35.835576 (+ 2123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:35.835582 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:35.835650 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:35.835768 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:35.837124 (+ 1356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:35.837130 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:35.837134 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:08:35.837155 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:35.837196 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:08:35.837211 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:35.837329 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:35.837495 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 15:08:35.837549 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 15:08:35.838709 9107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:35.832368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:42635 (local address 127.24.183.195:55657)
0809 15:08:35.833259 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:35.833337 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:35.835315 (+ 1978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:35.836052 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:35.836064 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:35.836181 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:35.836824 (+ 643us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:08:35.836826 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:35.836863 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:08:35.836964 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:08:35.837309 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:08:35.837318 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:08:35.837324 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 15:08:35.837570 (+ 246us) client_negotiation.cc:240] Negotiation successful
0809 15:08:35.837784 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":281,"threads_started":1}
I20260809 15:08:35.840049 9104 heartbeater.cc:347] Connected to a master server at 127.24.183.254:42635
I20260809 15:08:35.840337 9104 heartbeater.cc:464] Registering TS with master...
I20260809 15:08:35.840914 9104 heartbeater.cc:511] Master 127.24.183.254:42635 requested a full tablet report, sending...
I20260809 15:08:35.842348 8370 ts_manager.cc:194] Registered new tserver with Master: d59fe92dce8b4e17b3685df44fa33974 (127.24.183.195:41083)
I20260809 15:08:35.849929 25311 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 15:08:35.856375 8674 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:08:35.854865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53112 (local address 127.24.183.254:42635)
0809 15:08:35.855054 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:35.855060 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:35.855506 (+ 446us) server_negotiation.cc:407] Connection header received
0809 15:08:35.855629 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:35.855633 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:35.855706 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:35.855784 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:35.856135 (+ 351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53112: BlockingRecv error: recv got EOF from 127.0.0.1:53112 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 15:08:35.856771 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 8464
I20260809 15:08:35.860119 9117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:35.857686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47566 (local address 127.24.183.193:41167)
0809 15:08:35.858200 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:35.858207 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:35.858232 (+ 25us) server_negotiation.cc:407] Connection header received
0809 15:08:35.858287 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:35.858293 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:35.858493 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:35.858609 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:35.859467 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:35.859473 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:35.859483 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:08:35.859502 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:35.859540 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:08:35.859554 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:35.859656 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:35.859744 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 15:08:35.859828 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":244,"threads_started":1}
I20260809 15:08:35.861297 8663 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:08:35.942706 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 8464
I20260809 15:08:35.964404 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 8676
I20260809 15:08:35.967938 9120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:35.965380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51374 (local address 127.24.183.194:38771)
0809 15:08:35.965915 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:35.965922 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:35.965950 (+ 28us) server_negotiation.cc:407] Connection header received
0809 15:08:35.966015 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:35.966021 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:35.966249 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:35.966377 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:35.967194 (+ 817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:35.967203 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:35.967216 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:08:35.967235 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:35.967278 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:08:35.967292 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:35.967395 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:35.967477 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 15:08:35.967569 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":249,"threads_started":1}
I20260809 15:08:35.969159 8872 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:08:36.050856 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 8676
I20260809 15:08:36.074155 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 8884
I20260809 15:08:36.078217 9123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:36.075158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39088 (local address 127.24.183.195:41083)
0809 15:08:36.075863 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:36.075870 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:36.075901 (+ 31us) server_negotiation.cc:407] Connection header received
0809 15:08:36.075966 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:36.075973 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:36.076193 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:36.076343 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:36.077487 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:36.077494 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:36.077506 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:08:36.077523 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:36.077562 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:08:36.077576 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:36.077743 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:36.077820 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 15:08:36.077925 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":330,"threads_started":1}
I20260809 15:08:36.079465 9093 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:08:36.160986 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 8884
I20260809 15:08:36.183105 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 8318
I20260809 15:08:36.184342 8417 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:08:36.289635 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 8318
2026-08-09T15:08:36Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2397 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:08:36 dist-test-slave-nj21 krb5kdc[9140](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:08:36 dist-test-slave-nj21 krb5kdc[9140](info): set up 2 sockets
Aug 09 15:08:36 dist-test-slave-nj21 krb5kdc[9140](info): commencing operation
krb5kdc: starting...
W20260809 15:08:37.509116 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.000s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:08:37 dist-test-slave-nj21 krb5kdc[9140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:08:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:08:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
I20260809 15:08:37.778983 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:46515
--webserver_interface=127.24.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:38843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:46515
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:08:38.110476 9160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:08:38.110752 9160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:08:38.110898 9160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:08:38.118104 9160 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:08:38.118209 9160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:08:38.118283 9160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:08:38.118363 9160 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:08:38.118422 9160 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:08:38.128162 9160 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:38843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:46515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:46515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:08:38.129514 9160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:08:38.130854 9160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:08:38.142397 9166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:38.142441 9170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:38.142482 9173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:08:38.143195 9160 server_base.cc:1034] running on GCE node
I20260809 15:08:38.144047 9160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:08:38.145426 9160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:08:38.146648 9160 hybrid_clock.cc:648] HybridClock initialized: now 1786288118146588 us; error 39 us; skew 500 ppm
I20260809 15:08:38.147018 9160 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:08:38 dist-test-slave-nj21 krb5kdc[9140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288118, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:08:38.152881 9160 init.cc:377] Logged in from keytab as kudu/127.24.183.254@KRBTEST.COM (short username kudu)
I20260809 15:08:38.154307 9160 webserver.cc:460] Webserver started at http://127.24.183.254:40467/ using document root <none> and password file <none>
I20260809 15:08:38.154986 9160 fs_manager.cc:359] Metadata directory not provided
I20260809 15:08:38.155108 9160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:08:38.155402 9160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:08:38.158427 9160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "b93e826c76a44f59b380f1dfde5de3fa"
format_stamp: "Formatted at 2026-08-09 15:08:38 on dist-test-slave-nj21"
server_key: "7774f4cb13dab0d024e2b175dbabf65f"
server_key_iv: "2c58d79fdd7b66cffdadf6bf981e2ca3"
server_key_version: "encryptionkey@0"
I20260809 15:08:38.159174 9160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "b93e826c76a44f59b380f1dfde5de3fa"
format_stamp: "Formatted at 2026-08-09 15:08:38 on dist-test-slave-nj21"
server_key: "7774f4cb13dab0d024e2b175dbabf65f"
server_key_iv: "2c58d79fdd7b66cffdadf6bf981e2ca3"
server_key_version: "encryptionkey@0"
I20260809 15:08:38.164215 9160 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 15:08:38.167423 9184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:38.168644 9160 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260809 15:08:38.168814 9160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "b93e826c76a44f59b380f1dfde5de3fa"
format_stamp: "Formatted at 2026-08-09 15:08:38 on dist-test-slave-nj21"
server_key: "7774f4cb13dab0d024e2b175dbabf65f"
server_key_iv: "2c58d79fdd7b66cffdadf6bf981e2ca3"
server_key_version: "encryptionkey@0"
I20260809 15:08:38.168987 9160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:08:38.188194 9160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:08:38.192338 9160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:08:38.192596 9160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:08:38.210508 9160 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:46515
I20260809 15:08:38.210567 9270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:46515 every 8 connection(s)
I20260809 15:08:38.211915 9160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:08:38.213526 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 9160
I20260809 15:08:38.213774 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:08:38.214612 25311 external_mini_cluster.cc:1428] Setting key 5d5edee139f09afa0ec89b5ff181dc75
I20260809 15:08:38.216565 9272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 15:08:38 dist-test-slave-nj21 krb5kdc[9140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:08:38.229974 9272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b93e826c76a44f59b380f1dfde5de3fa: Bootstrap starting.
I20260809 15:08:38.233331 9272 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b93e826c76a44f59b380f1dfde5de3fa: Neither blocks nor log segments found. Creating new log.
I20260809 15:08:38.234583 9272 log.cc:824] T 00000000000000000000000000000000 P b93e826c76a44f59b380f1dfde5de3fa: Log is configured to *not* fsync() on all Append() calls
I20260809 15:08:38.237166 9276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:38.216615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40746 (local address 127.24.183.254:46515)
0809 15:08:38.217402 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:38.217413 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:38.217452 (+ 39us) server_negotiation.cc:407] Connection header received
0809 15:08:38.219004 (+ 1552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:38.219024 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:38.219392 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:38.219733 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:38.221557 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:38.222853 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:38.224613 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:38.225249 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:38.225387 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:38.230150 (+ 4763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:38.230161 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:38.230178 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:08:38.230215 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:38.233108 (+ 2893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:38.233919 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:38.233926 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:38.233932 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:38.234100 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:38.234492 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:38.234495 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:38.234497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:38.235045 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:38.235420 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:38.235837 (+ 417us) server_negotiation.cc:299] Negotiation successful
0809 15:08:38.236207 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":340,"threads_started":1}
I20260809 15:08:38.237679 9272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b93e826c76a44f59b380f1dfde5de3fa: No bootstrap required, opened a new log
I20260809 15:08:38.245411 9272 raft_consensus.cc:348] T 00000000000000000000000000000000 P b93e826c76a44f59b380f1dfde5de3fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b93e826c76a44f59b380f1dfde5de3fa" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 46515 } }
I20260809 15:08:38.245754 9272 raft_consensus.cc:374] T 00000000000000000000000000000000 P b93e826c76a44f59b380f1dfde5de3fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:08:38.245869 9272 raft_consensus.cc:729] T 00000000000000000000000000000000 P b93e826c76a44f59b380f1dfde5de3fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b93e826c76a44f59b380f1dfde5de3fa, State: Initialized, Role: FOLLOWER
I20260809 15:08:38.246505 9272 consensus_queue.cc:260] T 00000000000000000000000000000000 P b93e826c76a44f59b380f1dfde5de3fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b93e826c76a44f59b380f1dfde5de3fa" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 46515 } }
I20260809 15:08:38.246773 9272 raft_consensus.cc:388] T 00000000000000000000000000000000 P b93e826c76a44f59b380f1dfde5de3fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:08:38.246913 9272 raft_consensus.cc:482] T 00000000000000000000000000000000 P b93e826c76a44f59b380f1dfde5de3fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:08:38.247071 9272 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b93e826c76a44f59b380f1dfde5de3fa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:08:38.249545 9272 raft_consensus.cc:504] T 00000000000000000000000000000000 P b93e826c76a44f59b380f1dfde5de3fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b93e826c76a44f59b380f1dfde5de3fa" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 46515 } }
I20260809 15:08:38.250017 9272 leader_election.cc:302] T 00000000000000000000000000000000 P b93e826c76a44f59b380f1dfde5de3fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b93e826c76a44f59b380f1dfde5de3fa; no voters:
I20260809 15:08:38.250566 9272 leader_election.cc:288] T 00000000000000000000000000000000 P b93e826c76a44f59b380f1dfde5de3fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:08:38.250677 9279 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b93e826c76a44f59b380f1dfde5de3fa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:08:38.251319 9279 raft_consensus.cc:686] T 00000000000000000000000000000000 P b93e826c76a44f59b380f1dfde5de3fa [term 1 LEADER]: Becoming Leader. State: Replica: b93e826c76a44f59b380f1dfde5de3fa, State: Running, Role: LEADER
I20260809 15:08:38.251852 9279 consensus_queue.cc:237] T 00000000000000000000000000000000 P b93e826c76a44f59b380f1dfde5de3fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b93e826c76a44f59b380f1dfde5de3fa" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 46515 } }
I20260809 15:08:38.252553 9272 sys_catalog.cc:564] T 00000000000000000000000000000000 P b93e826c76a44f59b380f1dfde5de3fa [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:08:38.254907 9286 sys_catalog.cc:455] T 00000000000000000000000000000000 P b93e826c76a44f59b380f1dfde5de3fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b93e826c76a44f59b380f1dfde5de3fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b93e826c76a44f59b380f1dfde5de3fa" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 46515 } } }
I20260809 15:08:38.255012 9288 sys_catalog.cc:455] T 00000000000000000000000000000000 P b93e826c76a44f59b380f1dfde5de3fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader b93e826c76a44f59b380f1dfde5de3fa. Latest consensus state: current_term: 1 leader_uuid: "b93e826c76a44f59b380f1dfde5de3fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b93e826c76a44f59b380f1dfde5de3fa" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 46515 } } }
I20260809 15:08:38.255318 9286 sys_catalog.cc:458] T 00000000000000000000000000000000 P b93e826c76a44f59b380f1dfde5de3fa [sys.catalog]: This master's current role is: LEADER
I20260809 15:08:38.255343 9288 sys_catalog.cc:458] T 00000000000000000000000000000000 P b93e826c76a44f59b380f1dfde5de3fa [sys.catalog]: This master's current role is: LEADER
I20260809 15:08:38.257376 9294 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:08:38.262384 9294 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:08:38.270643 9294 catalog_manager.cc:1347] Generated new cluster ID: 4cc20172a5d24931bba6f92bd4e14525
I20260809 15:08:38.270756 9294 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:08:38.287370 9294 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:08:38.288846 9294 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:08:38.302378 9294 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b93e826c76a44f59b380f1dfde5de3fa: Generated new TSK 0
I20260809 15:08:38.303491 9294 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab.
Entry for principal kudu/127.24.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab.
I20260809 15:08:38.375166 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.193:0
--local_ip_for_outbound_sockets=127.24.183.193
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:46515
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:38843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:08:38.713653 9315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:08:38.713933 9315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:08:38.714057 9315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:08:38.721294 9315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:08:38.721396 9315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:08:38.721552 9315 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.193
I20260809 15:08:38.731002 9315 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:38843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:46515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.193
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:08:38.732326 9315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:08:38.733626 9315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:08:38.746579 9320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:38.746686 9321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:38.746723 9323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:08:38.747743 9315 server_base.cc:1034] running on GCE node
I20260809 15:08:38.748346 9315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:08:38.749293 9315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:08:38.750501 9315 hybrid_clock.cc:648] HybridClock initialized: now 1786288118750444 us; error 41 us; skew 500 ppm
I20260809 15:08:38.750825 9315 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:08:38 dist-test-slave-nj21 krb5kdc[9140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288118, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:08:38.754029 9315 init.cc:377] Logged in from keytab as kudu/127.24.183.193@KRBTEST.COM (short username kudu)
I20260809 15:08:38.755434 9315 webserver.cc:460] Webserver started at http://127.24.183.193:40393/ using document root <none> and password file <none>
I20260809 15:08:38.756146 9315 fs_manager.cc:359] Metadata directory not provided
I20260809 15:08:38.756276 9315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:08:38.756562 9315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:08:38.759495 9315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/data/instance:
uuid: "549a11c5d6d24eefa085ac838f8c4845"
format_stamp: "Formatted at 2026-08-09 15:08:38 on dist-test-slave-nj21"
server_key: "3d0f76ede8b2089c3bc3e5645e8ec172"
server_key_iv: "933ed3165ee391f7146db004f2d78574"
server_key_version: "encryptionkey@0"
I20260809 15:08:38.760336 9315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance:
uuid: "549a11c5d6d24eefa085ac838f8c4845"
format_stamp: "Formatted at 2026-08-09 15:08:38 on dist-test-slave-nj21"
server_key: "3d0f76ede8b2089c3bc3e5645e8ec172"
server_key_iv: "933ed3165ee391f7146db004f2d78574"
server_key_version: "encryptionkey@0"
I20260809 15:08:38.765542 9315 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 15:08:38.768828 9333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:38.770182 9315 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:08:38.770334 9315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/wal
uuid: "549a11c5d6d24eefa085ac838f8c4845"
format_stamp: "Formatted at 2026-08-09 15:08:38 on dist-test-slave-nj21"
server_key: "3d0f76ede8b2089c3bc3e5645e8ec172"
server_key_iv: "933ed3165ee391f7146db004f2d78574"
server_key_version: "encryptionkey@0"
I20260809 15:08:38.770506 9315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:08:38.803020 9315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:08:38.806982 9315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:08:38.807261 9315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:08:38.808256 9315 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:08:38.809614 9315 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:08:38.809692 9315 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:38.809830 9315 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:08:38.809886 9315 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:38.842343 9315 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.193:42483
I20260809 15:08:38.842374 9519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.193:42483 every 8 connection(s)
I20260809 15:08:38.843600 9315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
I20260809 15:08:38.851593 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 9315
I20260809 15:08:38.851846 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance
I20260809 15:08:38.852578 25311 external_mini_cluster.cc:1428] Setting key 17255cc7c29822b611e9cf4e74a4eb58
Aug 09 15:08:38 dist-test-slave-nj21 krb5kdc[9140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288118, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:08:38.863988 9526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:38.846057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:40227 (local address 127.24.183.254:46515)
0809 15:08:38.846507 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:38.846512 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:38.847435 (+ 923us) server_negotiation.cc:407] Connection header received
0809 15:08:38.849314 (+ 1879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:38.849320 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:38.849399 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:38.849508 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:38.850979 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:38.852031 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:38.853669 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:38.854145 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:38.854251 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:38.857998 (+ 3747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:38.858003 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:38.858006 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:08:38.858041 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:38.860911 (+ 2870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:38.861705 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:38.861709 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:38.861713 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:38.861835 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:38.862248 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:38.862252 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:38.862255 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:38.862539 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:38.862660 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:38.863432 (+ 772us) server_negotiation.cc:299] Negotiation successful
0809 15:08:38.863629 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":225,"threads_started":1}
I20260809 15:08:38.865181 9527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:38.846595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:46515 (local address 127.24.183.193:40227)
0809 15:08:38.847275 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:38.847316 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:38.849047 (+ 1731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:38.849748 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:38.849756 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:38.850259 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:38.850817 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:38.850834 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:38.852199 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:38.852204 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:38.853525 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:38.853542 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:38.854403 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:38.854412 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:38.854646 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:38.855253 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:38.855276 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:38.857789 (+ 2513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:08:38.861061 (+ 3272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:08:38.861069 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:08:38.861083 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:08:38.861611 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:08:38.861929 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:08:38.861933 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:08:38.861936 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:08:38.862160 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:08:38.862654 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:08:38.862661 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:08:38.863056 (+ 395us) client_negotiation.cc:769] Sending connection context
0809 15:08:38.863359 (+ 303us) client_negotiation.cc:240] Negotiation successful
0809 15:08:38.863880 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":276,"threads_started":1}
I20260809 15:08:38.866658 9523 heartbeater.cc:347] Connected to a master server at 127.24.183.254:46515
I20260809 15:08:38.867000 9523 heartbeater.cc:464] Registering TS with master...
I20260809 15:08:38.867820 9523 heartbeater.cc:511] Master 127.24.183.254:46515 requested a full tablet report, sending...
I20260809 15:08:38.870251 9205 ts_manager.cc:194] Registered new tserver with Master: 549a11c5d6d24eefa085ac838f8c4845 (127.24.183.193:42483)
I20260809 15:08:38.872279 9205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.193@KRBTEST.COM'} at 127.24.183.193:40227
WARNING: no policy specified for kudu/127.24.183.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab.
Entry for principal kudu/127.24.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab.
I20260809 15:08:38.915408 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.194:0
--local_ip_for_outbound_sockets=127.24.183.194
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:46515
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:38843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:08:39.246690 9535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:08:39.246963 9535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:08:39.247099 9535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:08:39.254416 9535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:08:39.254526 9535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:08:39.254676 9535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.194
I20260809 15:08:39.264680 9535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:38843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:46515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.194
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:08:39.266029 9535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:08:39.267349 9535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:08:39.280592 9544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:39.280627 9545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:39.280676 9550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:08:39.281740 9535 server_base.cc:1034] running on GCE node
I20260809 15:08:39.282354 9535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:08:39.283282 9535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:08:39.284487 9535 hybrid_clock.cc:648] HybridClock initialized: now 1786288119284435 us; error 48 us; skew 500 ppm
I20260809 15:08:39.284782 9535 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:08:39 dist-test-slave-nj21 krb5kdc[9140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288119, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:08:39.288012 9535 init.cc:377] Logged in from keytab as kudu/127.24.183.194@KRBTEST.COM (short username kudu)
I20260809 15:08:39.289400 9535 webserver.cc:460] Webserver started at http://127.24.183.194:37453/ using document root <none> and password file <none>
I20260809 15:08:39.290078 9535 fs_manager.cc:359] Metadata directory not provided
I20260809 15:08:39.290196 9535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:08:39.290490 9535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:08:39.293457 9535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/data/instance:
uuid: "eb84f2b6e8ee42b583570decf843e6c7"
format_stamp: "Formatted at 2026-08-09 15:08:39 on dist-test-slave-nj21"
server_key: "2796ae6eefd0f4ed9658139c9d3e7a6b"
server_key_iv: "32e87242df503870f5dbe202869852e4"
server_key_version: "encryptionkey@0"
I20260809 15:08:39.294229 9535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance:
uuid: "eb84f2b6e8ee42b583570decf843e6c7"
format_stamp: "Formatted at 2026-08-09 15:08:39 on dist-test-slave-nj21"
server_key: "2796ae6eefd0f4ed9658139c9d3e7a6b"
server_key_iv: "32e87242df503870f5dbe202869852e4"
server_key_version: "encryptionkey@0"
I20260809 15:08:39.299244 9535 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 15:08:39.302659 9562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:39.304026 9535 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:08:39.304181 9535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/wal
uuid: "eb84f2b6e8ee42b583570decf843e6c7"
format_stamp: "Formatted at 2026-08-09 15:08:39 on dist-test-slave-nj21"
server_key: "2796ae6eefd0f4ed9658139c9d3e7a6b"
server_key_iv: "32e87242df503870f5dbe202869852e4"
server_key_version: "encryptionkey@0"
I20260809 15:08:39.304373 9535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:08:39.324613 9535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:08:39.328675 9535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:08:39.328981 9535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:08:39.330031 9535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:08:39.331518 9535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:08:39.331602 9535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:39.331768 9535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:08:39.331830 9535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:39.365569 9535 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.194:39385
I20260809 15:08:39.365615 9763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.194:39385 every 8 connection(s)
I20260809 15:08:39.366914 9535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
I20260809 15:08:39.370749 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 9535
I20260809 15:08:39.370982 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance
I20260809 15:08:39.371539 25311 external_mini_cluster.cc:1428] Setting key 0dbc8444c5fadec7bc7239b6b7145041
Aug 09 15:08:39 dist-test-slave-nj21 krb5kdc[9140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288119, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:08:39.386632 9767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:39.369488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:52315 (local address 127.24.183.254:46515)
0809 15:08:39.369931 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:39.369936 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:39.371746 (+ 1810us) server_negotiation.cc:407] Connection header received
0809 15:08:39.373144 (+ 1398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:39.373151 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:39.373236 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:39.373366 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:39.374971 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:39.375775 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:39.377270 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:39.377706 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:39.377817 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:39.381098 (+ 3281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:39.381103 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:39.381107 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:08:39.381139 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:39.383691 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:39.384377 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:39.384383 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:39.384386 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:39.384518 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:39.384925 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:39.384930 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:39.384934 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:39.385219 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:39.385355 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:39.386032 (+ 677us) server_negotiation.cc:299] Negotiation successful
0809 15:08:39.386262 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":225,"threads_started":1}
I20260809 15:08:39.387606 9770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:39.369951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:46515 (local address 127.24.183.194:52315)
0809 15:08:39.370763 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:39.370821 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:39.372889 (+ 2068us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:39.373583 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:39.373594 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:39.374190 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:39.374804 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:39.374824 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:39.375920 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:39.375924 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:39.377130 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:39.377143 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:39.377929 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:39.377936 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:39.378156 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:39.378761 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:39.378791 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:39.380886 (+ 2095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:08:39.383829 (+ 2943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:08:39.383839 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:08:39.383856 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:08:39.384265 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:08:39.384622 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:08:39.384627 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:08:39.384631 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:08:39.384819 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:08:39.385353 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:08:39.385361 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:08:39.385686 (+ 325us) client_negotiation.cc:769] Sending connection context
0809 15:08:39.385925 (+ 239us) client_negotiation.cc:240] Negotiation successful
0809 15:08:39.386459 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":278,"threads_started":1}
I20260809 15:08:39.389050 9764 heartbeater.cc:347] Connected to a master server at 127.24.183.254:46515
I20260809 15:08:39.389490 9764 heartbeater.cc:464] Registering TS with master...
I20260809 15:08:39.390463 9764 heartbeater.cc:511] Master 127.24.183.254:46515 requested a full tablet report, sending...
I20260809 15:08:39.392179 9205 ts_manager.cc:194] Registered new tserver with Master: eb84f2b6e8ee42b583570decf843e6c7 (127.24.183.194:39385)
I20260809 15:08:39.393008 9205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.194@KRBTEST.COM'} at 127.24.183.194:52315
WARNING: no policy specified for kudu/127.24.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab.
Entry for principal kudu/127.24.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab.
I20260809 15:08:39.433084 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.195:0
--local_ip_for_outbound_sockets=127.24.183.195
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:46515
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:38843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:08:39.767321 9773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:08:39.767606 9773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:08:39.767771 9773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:08:39.775394 9773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:08:39.775508 9773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:08:39.775660 9773 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.195
I20260809 15:08:39.785471 9773 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:38843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:46515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.195
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:08:39.786774 9773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:08:39.788141 9773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:08:39.800910 9785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:39.800956 9789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:08:39.800879 9787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:08:39.801604 9773 server_base.cc:1034] running on GCE node
I20260809 15:08:39.802158 9773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:08:39.803030 9773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:08:39.804226 9773 hybrid_clock.cc:648] HybridClock initialized: now 1786288119804174 us; error 43 us; skew 500 ppm
I20260809 15:08:39.804525 9773 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:08:39 dist-test-slave-nj21 krb5kdc[9140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288119, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:08:39.807624 9773 init.cc:377] Logged in from keytab as kudu/127.24.183.195@KRBTEST.COM (short username kudu)
I20260809 15:08:39.809034 9773 webserver.cc:460] Webserver started at http://127.24.183.195:33787/ using document root <none> and password file <none>
I20260809 15:08:39.809686 9773 fs_manager.cc:359] Metadata directory not provided
I20260809 15:08:39.809806 9773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:08:39.810086 9773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:08:39.812938 9773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/data/instance:
uuid: "629e64d41cef4647a1da583794849278"
format_stamp: "Formatted at 2026-08-09 15:08:39 on dist-test-slave-nj21"
server_key: "e162fa06d92bbc539f08634022343752"
server_key_iv: "fcab2064ac4a1737376947a50a223509"
server_key_version: "encryptionkey@0"
I20260809 15:08:39.813679 9773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance:
uuid: "629e64d41cef4647a1da583794849278"
format_stamp: "Formatted at 2026-08-09 15:08:39 on dist-test-slave-nj21"
server_key: "e162fa06d92bbc539f08634022343752"
server_key_iv: "fcab2064ac4a1737376947a50a223509"
server_key_version: "encryptionkey@0"
I20260809 15:08:39.818589 9773 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 15:08:39.821852 9800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:39.823109 9773 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 15:08:39.823263 9773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/wal
uuid: "629e64d41cef4647a1da583794849278"
format_stamp: "Formatted at 2026-08-09 15:08:39 on dist-test-slave-nj21"
server_key: "e162fa06d92bbc539f08634022343752"
server_key_iv: "fcab2064ac4a1737376947a50a223509"
server_key_version: "encryptionkey@0"
I20260809 15:08:39.823446 9773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:08:39.835192 9773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:08:39.839138 9773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:08:39.839411 9773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:08:39.840430 9773 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:08:39.841734 9773 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:08:39.841814 9773 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:39.841967 9773 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:08:39.842028 9773 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:08:39.873188 9773 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.195:43881
I20260809 15:08:39.873229 9974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.195:43881 every 8 connection(s)
I20260809 15:08:39.874469 9773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
I20260809 15:08:39.875823 9523 heartbeater.cc:503] Master 127.24.183.254:46515 was elected leader, sending a full tablet report...
I20260809 15:08:39.879647 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 9773
I20260809 15:08:39.879859 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance
I20260809 15:08:39.880304 25311 external_mini_cluster.cc:1428] Setting key cb48d02cf3019679b522496a081e1d78
Aug 09 15:08:39 dist-test-slave-nj21 krb5kdc[9140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288119, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:08:39.893935 9767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:39.877192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:57707 (local address 127.24.183.254:46515)
0809 15:08:39.877362 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:39.877367 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:39.878637 (+ 1270us) server_negotiation.cc:407] Connection header received
0809 15:08:39.880513 (+ 1876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:39.880519 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:39.880613 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:39.880700 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:39.882413 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:39.883237 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:39.884775 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:39.885177 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:39.885298 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:39.888588 (+ 3290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:39.888593 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:39.888597 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:08:39.888630 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:39.891047 (+ 2417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:39.891781 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:39.891786 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:39.891788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:39.891921 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:39.892286 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:39.892289 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:39.892291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:39.892561 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:39.892681 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:39.893372 (+ 691us) server_negotiation.cc:299] Negotiation successful
0809 15:08:39.893597 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 15:08:39.894910 9981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:39.877803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:46515 (local address 127.24.183.195:57707)
0809 15:08:39.878496 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:39.878533 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:39.880261 (+ 1728us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:39.880947 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:39.880959 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:39.881603 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:08:39.882259 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:39.882279 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:39.883382 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:39.883386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:39.884634 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:39.884651 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:39.885400 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:39.885407 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:39.885628 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:39.886236 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:08:39.886258 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:08:39.888387 (+ 2129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:08:39.891188 (+ 2801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:08:39.891195 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:08:39.891207 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:08:39.891611 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:08:39.892017 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:08:39.892020 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:08:39.892022 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:08:39.892196 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:08:39.892654 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:08:39.892660 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:08:39.892951 (+ 291us) client_negotiation.cc:769] Sending connection context
0809 15:08:39.893170 (+ 219us) client_negotiation.cc:240] Negotiation successful
0809 15:08:39.893638 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":257,"threads_started":1}
I20260809 15:08:39.896293 9977 heartbeater.cc:347] Connected to a master server at 127.24.183.254:46515
I20260809 15:08:39.896611 9977 heartbeater.cc:464] Registering TS with master...
I20260809 15:08:39.897329 9977 heartbeater.cc:511] Master 127.24.183.254:46515 requested a full tablet report, sending...
I20260809 15:08:39.898681 9205 ts_manager.cc:194] Registered new tserver with Master: 629e64d41cef4647a1da583794849278 (127.24.183.195:43881)
I20260809 15:08:39.899493 9205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.195@KRBTEST.COM'} at 127.24.183.195:57707
I20260809 15:08:39.907696 25311 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:08:39.924252 9767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:39.912122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40752 (local address 127.24.183.254:46515)
0809 15:08:39.912325 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:39.912332 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:39.912750 (+ 418us) server_negotiation.cc:407] Connection header received
0809 15:08:39.912890 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:39.912894 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:39.912964 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:39.913037 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:39.914570 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:39.915418 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:39.916693 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:39.917134 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:39.917247 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:39.919549 (+ 2302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:39.919554 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:39.919557 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:08:39.919588 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:39.921919 (+ 2331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:39.922488 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:39.922491 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:39.922494 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:39.922600 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:39.922866 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:39.922869 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:39.922871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:39.923166 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:39.923360 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:39.923728 (+ 368us) server_negotiation.cc:299] Negotiation successful
0809 15:08:39.923930 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 15:08:39.928018 9205 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40752:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:08:39.934744 9205 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:08:39.958401 10005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:39.947404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45816 (local address 127.24.183.193:42483)
0809 15:08:39.949550 (+ 2146us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:39.949559 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:39.950248 (+ 689us) server_negotiation.cc:407] Connection header received
0809 15:08:39.950754 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:39.950763 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:39.951008 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:39.951191 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:08:39.952233 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:39.953679 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:39.956248 (+ 2569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:39.957364 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:39.957551 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:39.957729 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:39.957986 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 15:08:39.958156 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1957,"thread_start_us":1770,"threads_started":1}
I20260809 15:08:39.959053 10008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:39.949493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36082 (local address 127.24.183.194:39385)
0809 15:08:39.950710 (+ 1217us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:39.950717 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:39.950741 (+ 24us) server_negotiation.cc:407] Connection header received
0809 15:08:39.950824 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:39.950830 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:39.951004 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:39.951191 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:08:39.952194 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:39.953458 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:39.956203 (+ 2745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:39.957699 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:39.957852 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:39.958023 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:39.958551 (+ 528us) server_negotiation.cc:299] Negotiation successful
0809 15:08:39.958725 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1065,"thread_start_us":934,"threads_started":1}
I20260809 15:08:39.959306 10003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:39.947194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56848 (local address 127.24.183.195:43881)
0809 15:08:39.948216 (+ 1022us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:39.948225 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:39.948260 (+ 35us) server_negotiation.cc:407] Connection header received
0809 15:08:39.948359 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:39.948366 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:39.948612 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:39.948805 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:08:39.951221 (+ 2416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:39.952637 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:39.955329 (+ 2692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:39.956442 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:39.956633 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:39.956820 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:39.958821 (+ 2001us) server_negotiation.cc:299] Negotiation successful
0809 15:08:39.959011 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":856,"thread_start_us":1092,"threads_started":1}
I20260809 15:08:39.958015 10001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:39.946600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:43881 (local address 127.0.0.1:56848)
0809 15:08:39.947397 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:39.947435 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:39.947624 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:39.948868 (+ 1244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:39.948877 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:39.948912 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:08:39.950987 (+ 2075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:39.951007 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:39.952800 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:39.952806 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:39.955175 (+ 2369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:39.955187 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:39.956955 (+ 1768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:39.956962 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:39.957156 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:39.957208 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 15:08:39.957339 (+ 131us) client_negotiation.cc:240] Negotiation successful
0809 15:08:39.957539 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"mutex_wait_us":27,"thread_start_us":1061,"threads_started":1}
I20260809 15:08:39.960073 10006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:39.949242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:39385 (local address 127.0.0.1:36082)
0809 15:08:39.950328 (+ 1086us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:39.950346 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:39.950457 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:39.951433 (+ 976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:39.951438 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:39.951480 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:08:39.951968 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:39.951980 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:39.953584 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:39.953589 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:39.956017 (+ 2428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:39.956033 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:39.958170 (+ 2137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:39.958176 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:39.958344 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:39.958388 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 15:08:39.958475 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 15:08:39.958585 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":951,"mutex_wait_us":63,"spinlock_wait_cycles":5120,"thread_start_us":791,"threads_started":1}
I20260809 15:08:39.960119 10002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:39.946970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:42483 (local address 127.0.0.1:45816)
0809 15:08:39.949997 (+ 3027us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:39.950017 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:39.950598 (+ 581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:39.951432 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:39.951438 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:39.951470 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:08:39.952049 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:39.952058 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:39.953819 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:39.953825 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:39.956079 (+ 2254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:39.956090 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:39.957606 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:39.957613 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:39.957793 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:39.957827 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 15:08:39.957903 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 15:08:39.958090 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2879,"mutex_wait_us":8,"spinlock_wait_cycles":998528,"thread_start_us":2726,"threads_started":1}
I20260809 15:08:39.963513 9413 tablet_service.cc:1467] Processing CreateTablet for tablet ef532e6d7d604ed7bf7c59dcc85025e6 (DEFAULT_TABLE table=test-table [id=aca9430f4bbd498c9bfb47e35adc6b8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:08:39.963510 9870 tablet_service.cc:1467] Processing CreateTablet for tablet ef532e6d7d604ed7bf7c59dcc85025e6 (DEFAULT_TABLE table=test-table [id=aca9430f4bbd498c9bfb47e35adc6b8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:08:39.963912 9661 tablet_service.cc:1467] Processing CreateTablet for tablet ef532e6d7d604ed7bf7c59dcc85025e6 (DEFAULT_TABLE table=test-table [id=aca9430f4bbd498c9bfb47e35adc6b8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:08:39.964936 9413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef532e6d7d604ed7bf7c59dcc85025e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:08:39.965018 9870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef532e6d7d604ed7bf7c59dcc85025e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:08:39.965096 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef532e6d7d604ed7bf7c59dcc85025e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:08:39.979411 10012 tablet_bootstrap.cc:492] T ef532e6d7d604ed7bf7c59dcc85025e6 P eb84f2b6e8ee42b583570decf843e6c7: Bootstrap starting.
I20260809 15:08:39.980736 10011 tablet_bootstrap.cc:492] T ef532e6d7d604ed7bf7c59dcc85025e6 P 629e64d41cef4647a1da583794849278: Bootstrap starting.
I20260809 15:08:39.982646 10009 tablet_bootstrap.cc:492] T ef532e6d7d604ed7bf7c59dcc85025e6 P 549a11c5d6d24eefa085ac838f8c4845: Bootstrap starting.
I20260809 15:08:39.984663 10011 tablet_bootstrap.cc:654] T ef532e6d7d604ed7bf7c59dcc85025e6 P 629e64d41cef4647a1da583794849278: Neither blocks nor log segments found. Creating new log.
I20260809 15:08:39.986156 10009 tablet_bootstrap.cc:654] T ef532e6d7d604ed7bf7c59dcc85025e6 P 549a11c5d6d24eefa085ac838f8c4845: Neither blocks nor log segments found. Creating new log.
I20260809 15:08:39.986462 10011 log.cc:824] T ef532e6d7d604ed7bf7c59dcc85025e6 P 629e64d41cef4647a1da583794849278: Log is configured to *not* fsync() on all Append() calls
I20260809 15:08:39.986513 10012 tablet_bootstrap.cc:654] T ef532e6d7d604ed7bf7c59dcc85025e6 P eb84f2b6e8ee42b583570decf843e6c7: Neither blocks nor log segments found. Creating new log.
I20260809 15:08:39.987629 10012 log.cc:824] T ef532e6d7d604ed7bf7c59dcc85025e6 P eb84f2b6e8ee42b583570decf843e6c7: Log is configured to *not* fsync() on all Append() calls
I20260809 15:08:39.988513 10009 log.cc:824] T ef532e6d7d604ed7bf7c59dcc85025e6 P 549a11c5d6d24eefa085ac838f8c4845: Log is configured to *not* fsync() on all Append() calls
I20260809 15:08:39.989723 10011 tablet_bootstrap.cc:492] T ef532e6d7d604ed7bf7c59dcc85025e6 P 629e64d41cef4647a1da583794849278: No bootstrap required, opened a new log
I20260809 15:08:39.989987 10011 ts_tablet_manager.cc:1397] T ef532e6d7d604ed7bf7c59dcc85025e6 P 629e64d41cef4647a1da583794849278: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.006s
I20260809 15:08:39.990342 10012 tablet_bootstrap.cc:492] T ef532e6d7d604ed7bf7c59dcc85025e6 P eb84f2b6e8ee42b583570decf843e6c7: No bootstrap required, opened a new log
I20260809 15:08:39.990579 10012 ts_tablet_manager.cc:1397] T ef532e6d7d604ed7bf7c59dcc85025e6 P eb84f2b6e8ee42b583570decf843e6c7: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
I20260809 15:08:39.991508 10009 tablet_bootstrap.cc:492] T ef532e6d7d604ed7bf7c59dcc85025e6 P 549a11c5d6d24eefa085ac838f8c4845: No bootstrap required, opened a new log
I20260809 15:08:39.991799 10009 ts_tablet_manager.cc:1397] T ef532e6d7d604ed7bf7c59dcc85025e6 P 549a11c5d6d24eefa085ac838f8c4845: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260809 15:08:39.997743 10012 raft_consensus.cc:348] T ef532e6d7d604ed7bf7c59dcc85025e6 P eb84f2b6e8ee42b583570decf843e6c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629e64d41cef4647a1da583794849278" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 43881 } } peers { permanent_uuid: "549a11c5d6d24eefa085ac838f8c4845" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 42483 } } peers { permanent_uuid: "eb84f2b6e8ee42b583570decf843e6c7" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 39385 } }
I20260809 15:08:39.998093 10012 raft_consensus.cc:374] T ef532e6d7d604ed7bf7c59dcc85025e6 P eb84f2b6e8ee42b583570decf843e6c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:08:39.998224 10012 raft_consensus.cc:729] T ef532e6d7d604ed7bf7c59dcc85025e6 P eb84f2b6e8ee42b583570decf843e6c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb84f2b6e8ee42b583570decf843e6c7, State: Initialized, Role: FOLLOWER
I20260809 15:08:39.998793 10012 consensus_queue.cc:260] T ef532e6d7d604ed7bf7c59dcc85025e6 P eb84f2b6e8ee42b583570decf843e6c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629e64d41cef4647a1da583794849278" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 43881 } } peers { permanent_uuid: "549a11c5d6d24eefa085ac838f8c4845" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 42483 } } peers { permanent_uuid: "eb84f2b6e8ee42b583570decf843e6c7" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 39385 } }
I20260809 15:08:39.999939 9764 heartbeater.cc:503] Master 127.24.183.254:46515 was elected leader, sending a full tablet report...
I20260809 15:08:40.000413 10012 ts_tablet_manager.cc:1428] T ef532e6d7d604ed7bf7c59dcc85025e6 P eb84f2b6e8ee42b583570decf843e6c7: Time spent starting tablet: real 0.010s user 0.006s sys 0.005s
I20260809 15:08:40.001132 10011 raft_consensus.cc:348] T ef532e6d7d604ed7bf7c59dcc85025e6 P 629e64d41cef4647a1da583794849278 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629e64d41cef4647a1da583794849278" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 43881 } } peers { permanent_uuid: "549a11c5d6d24eefa085ac838f8c4845" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 42483 } } peers { permanent_uuid: "eb84f2b6e8ee42b583570decf843e6c7" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 39385 } }
I20260809 15:08:40.001566 10011 raft_consensus.cc:374] T ef532e6d7d604ed7bf7c59dcc85025e6 P 629e64d41cef4647a1da583794849278 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:08:40.001665 10011 raft_consensus.cc:729] T ef532e6d7d604ed7bf7c59dcc85025e6 P 629e64d41cef4647a1da583794849278 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 629e64d41cef4647a1da583794849278, State: Initialized, Role: FOLLOWER
I20260809 15:08:40.002269 10011 consensus_queue.cc:260] T ef532e6d7d604ed7bf7c59dcc85025e6 P 629e64d41cef4647a1da583794849278 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629e64d41cef4647a1da583794849278" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 43881 } } peers { permanent_uuid: "549a11c5d6d24eefa085ac838f8c4845" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 42483 } } peers { permanent_uuid: "eb84f2b6e8ee42b583570decf843e6c7" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 39385 } }
I20260809 15:08:40.003350 9977 heartbeater.cc:503] Master 127.24.183.254:46515 was elected leader, sending a full tablet report...
I20260809 15:08:40.003916 10009 raft_consensus.cc:348] T ef532e6d7d604ed7bf7c59dcc85025e6 P 549a11c5d6d24eefa085ac838f8c4845 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629e64d41cef4647a1da583794849278" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 43881 } } peers { permanent_uuid: "549a11c5d6d24eefa085ac838f8c4845" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 42483 } } peers { permanent_uuid: "eb84f2b6e8ee42b583570decf843e6c7" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 39385 } }
I20260809 15:08:40.004328 10009 raft_consensus.cc:374] T ef532e6d7d604ed7bf7c59dcc85025e6 P 549a11c5d6d24eefa085ac838f8c4845 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:08:40.004431 10009 raft_consensus.cc:729] T ef532e6d7d604ed7bf7c59dcc85025e6 P 549a11c5d6d24eefa085ac838f8c4845 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 549a11c5d6d24eefa085ac838f8c4845, State: Initialized, Role: FOLLOWER
I20260809 15:08:40.005335 10011 ts_tablet_manager.cc:1428] T ef532e6d7d604ed7bf7c59dcc85025e6 P 629e64d41cef4647a1da583794849278: Time spent starting tablet: real 0.015s user 0.012s sys 0.000s
I20260809 15:08:40.005769 10017 raft_consensus.cc:482] T ef532e6d7d604ed7bf7c59dcc85025e6 P eb84f2b6e8ee42b583570decf843e6c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:08:40.006079 10017 raft_consensus.cc:504] T ef532e6d7d604ed7bf7c59dcc85025e6 P eb84f2b6e8ee42b583570decf843e6c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629e64d41cef4647a1da583794849278" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 43881 } } peers { permanent_uuid: "549a11c5d6d24eefa085ac838f8c4845" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 42483 } } peers { permanent_uuid: "eb84f2b6e8ee42b583570decf843e6c7" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 39385 } }
I20260809 15:08:40.006276 10009 consensus_queue.cc:260] T ef532e6d7d604ed7bf7c59dcc85025e6 P 549a11c5d6d24eefa085ac838f8c4845 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629e64d41cef4647a1da583794849278" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 43881 } } peers { permanent_uuid: "549a11c5d6d24eefa085ac838f8c4845" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 42483 } } peers { permanent_uuid: "eb84f2b6e8ee42b583570decf843e6c7" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 39385 } }
I20260809 15:08:40.009752 10017 leader_election.cc:288] T ef532e6d7d604ed7bf7c59dcc85025e6 P eb84f2b6e8ee42b583570decf843e6c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 629e64d41cef4647a1da583794849278 (127.24.183.195:43881), 549a11c5d6d24eefa085ac838f8c4845 (127.24.183.193:42483)
I20260809 15:08:40.008755 10009 ts_tablet_manager.cc:1428] T ef532e6d7d604ed7bf7c59dcc85025e6 P 549a11c5d6d24eefa085ac838f8c4845: Time spent starting tablet: real 0.017s user 0.013s sys 0.000s
I20260809 15:08:40.016227 10003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:40.009273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:53213 (local address 127.24.183.195:43881)
0809 15:08:40.009604 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:40.009610 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:40.009630 (+ 20us) server_negotiation.cc:407] Connection header received
0809 15:08:40.009828 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:40.009833 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:40.009942 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:40.010027 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:08:40.010770 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:40.011875 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:40.014264 (+ 2389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:40.015396 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:40.015513 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:40.015565 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:40.015939 (+ 374us) server_negotiation.cc:299] Negotiation successful
0809 15:08:40.016032 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194}
I20260809 15:08:40.016409 10026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:40.008848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:43881 (local address 127.24.183.194:53213)
0809 15:08:40.009520 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:40.009546 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:40.009691 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:40.010087 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:40.010092 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:40.010125 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:08:40.010577 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:40.010593 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:40.012585 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:40.012591 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:40.014109 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:40.014121 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:40.015592 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:40.015598 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:40.015790 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:40.015827 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 15:08:40.015925 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 15:08:40.016059 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":485,"threads_started":1}
I20260809 15:08:40.017295 10028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:40.010130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:42483 (local address 127.24.183.194:48335)
0809 15:08:40.010872 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:40.010889 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:08:40.010975 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:08:40.011949 (+ 974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:08:40.011954 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:08:40.011979 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:08:40.012302 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:40.012310 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:40.013335 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:40.013341 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:40.015012 (+ 1671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:08:40.015028 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:08:40.016604 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:08:40.016611 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:08:40.016802 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:40.016836 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 15:08:40.016920 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 15:08:40.017014 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":798,"threads_started":1}
I20260809 15:08:40.017335 10005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:40.010756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:48335 (local address 127.24.183.193:42483)
0809 15:08:40.011505 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:40.011512 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:40.011530 (+ 18us) server_negotiation.cc:407] Connection header received
0809 15:08:40.011590 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:40.011595 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:40.011656 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:40.011789 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:08:40.012445 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:40.013194 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:40.015180 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:40.016299 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:40.016451 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:40.016521 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:40.016980 (+ 459us) server_negotiation.cc:299] Negotiation successful
0809 15:08:40.017113 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614}
I20260809 15:08:40.017378 9908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef532e6d7d604ed7bf7c59dcc85025e6" candidate_uuid: "eb84f2b6e8ee42b583570decf843e6c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "629e64d41cef4647a1da583794849278" is_pre_election: true
I20260809 15:08:40.017858 9908 raft_consensus.cc:2454] T ef532e6d7d604ed7bf7c59dcc85025e6 P 629e64d41cef4647a1da583794849278 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb84f2b6e8ee42b583570decf843e6c7 in term 0.
I20260809 15:08:40.018215 9446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef532e6d7d604ed7bf7c59dcc85025e6" candidate_uuid: "eb84f2b6e8ee42b583570decf843e6c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "549a11c5d6d24eefa085ac838f8c4845" is_pre_election: true
I20260809 15:08:40.018456 9568 leader_election.cc:302] T ef532e6d7d604ed7bf7c59dcc85025e6 P eb84f2b6e8ee42b583570decf843e6c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 629e64d41cef4647a1da583794849278, eb84f2b6e8ee42b583570decf843e6c7; no voters:
I20260809 15:08:40.018708 9446 raft_consensus.cc:2454] T ef532e6d7d604ed7bf7c59dcc85025e6 P 549a11c5d6d24eefa085ac838f8c4845 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb84f2b6e8ee42b583570decf843e6c7 in term 0.
I20260809 15:08:40.018823 10017 raft_consensus.cc:2781] T ef532e6d7d604ed7bf7c59dcc85025e6 P eb84f2b6e8ee42b583570decf843e6c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:08:40.018970 10017 raft_consensus.cc:482] T ef532e6d7d604ed7bf7c59dcc85025e6 P eb84f2b6e8ee42b583570decf843e6c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:08:40.019084 10017 raft_consensus.cc:3037] T ef532e6d7d604ed7bf7c59dcc85025e6 P eb84f2b6e8ee42b583570decf843e6c7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:08:40.021437 10017 raft_consensus.cc:504] T ef532e6d7d604ed7bf7c59dcc85025e6 P eb84f2b6e8ee42b583570decf843e6c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629e64d41cef4647a1da583794849278" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 43881 } } peers { permanent_uuid: "549a11c5d6d24eefa085ac838f8c4845" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 42483 } } peers { permanent_uuid: "eb84f2b6e8ee42b583570decf843e6c7" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 39385 } }
I20260809 15:08:40.021997 10017 leader_election.cc:288] T ef532e6d7d604ed7bf7c59dcc85025e6 P eb84f2b6e8ee42b583570decf843e6c7 [CANDIDATE]: Term 1 election: Requested vote from peers 629e64d41cef4647a1da583794849278 (127.24.183.195:43881), 549a11c5d6d24eefa085ac838f8c4845 (127.24.183.193:42483)
I20260809 15:08:40.022341 9908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef532e6d7d604ed7bf7c59dcc85025e6" candidate_uuid: "eb84f2b6e8ee42b583570decf843e6c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "629e64d41cef4647a1da583794849278"
I20260809 15:08:40.022455 9446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef532e6d7d604ed7bf7c59dcc85025e6" candidate_uuid: "eb84f2b6e8ee42b583570decf843e6c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "549a11c5d6d24eefa085ac838f8c4845"
I20260809 15:08:40.022521 9908 raft_consensus.cc:3037] T ef532e6d7d604ed7bf7c59dcc85025e6 P 629e64d41cef4647a1da583794849278 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:08:40.022639 9446 raft_consensus.cc:3037] T ef532e6d7d604ed7bf7c59dcc85025e6 P 549a11c5d6d24eefa085ac838f8c4845 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:08:40.024986 9446 raft_consensus.cc:2454] T ef532e6d7d604ed7bf7c59dcc85025e6 P 549a11c5d6d24eefa085ac838f8c4845 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb84f2b6e8ee42b583570decf843e6c7 in term 1.
I20260809 15:08:40.025018 9908 raft_consensus.cc:2454] T ef532e6d7d604ed7bf7c59dcc85025e6 P 629e64d41cef4647a1da583794849278 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb84f2b6e8ee42b583570decf843e6c7 in term 1.
I20260809 15:08:40.025424 9568 leader_election.cc:302] T ef532e6d7d604ed7bf7c59dcc85025e6 P eb84f2b6e8ee42b583570decf843e6c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 549a11c5d6d24eefa085ac838f8c4845, 629e64d41cef4647a1da583794849278, eb84f2b6e8ee42b583570decf843e6c7; no voters:
I20260809 15:08:40.025731 10017 raft_consensus.cc:2781] T ef532e6d7d604ed7bf7c59dcc85025e6 P eb84f2b6e8ee42b583570decf843e6c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:08:40.026120 10017 raft_consensus.cc:686] T ef532e6d7d604ed7bf7c59dcc85025e6 P eb84f2b6e8ee42b583570decf843e6c7 [term 1 LEADER]: Becoming Leader. State: Replica: eb84f2b6e8ee42b583570decf843e6c7, State: Running, Role: LEADER
I20260809 15:08:40.026593 10017 consensus_queue.cc:237] T ef532e6d7d604ed7bf7c59dcc85025e6 P eb84f2b6e8ee42b583570decf843e6c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629e64d41cef4647a1da583794849278" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 43881 } } peers { permanent_uuid: "549a11c5d6d24eefa085ac838f8c4845" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 42483 } } peers { permanent_uuid: "eb84f2b6e8ee42b583570decf843e6c7" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 39385 } }
I20260809 15:08:40.031246 9204 catalog_manager.cc:5495] T ef532e6d7d604ed7bf7c59dcc85025e6 P eb84f2b6e8ee42b583570decf843e6c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to eb84f2b6e8ee42b583570decf843e6c7 (127.24.183.194). New cstate: current_term: 1 leader_uuid: "eb84f2b6e8ee42b583570decf843e6c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "629e64d41cef4647a1da583794849278" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 43881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "549a11c5d6d24eefa085ac838f8c4845" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 42483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb84f2b6e8ee42b583570decf843e6c7" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 39385 } health_report { overall_health: HEALTHY } } }
I20260809 15:08:40.056622 10008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:40.051429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36084 (local address 127.24.183.194:39385)
0809 15:08:40.051650 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:40.051657 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:40.051764 (+ 107us) server_negotiation.cc:407] Connection header received
0809 15:08:40.051976 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:40.051980 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:40.052058 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:40.052213 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:08:40.052605 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:40.053425 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:40.054667 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:40.055073 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:40.055170 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:40.055551 (+ 381us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:08:40.056113 (+ 562us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:08:40.056229 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:40.056293 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 15:08:40.056361 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 15:08:40.064429 9446 raft_consensus.cc:1261] T ef532e6d7d604ed7bf7c59dcc85025e6 P 549a11c5d6d24eefa085ac838f8c4845 [term 1 FOLLOWER]: Refusing update from remote peer eb84f2b6e8ee42b583570decf843e6c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:08:40.064429 9908 raft_consensus.cc:1261] T ef532e6d7d604ed7bf7c59dcc85025e6 P 629e64d41cef4647a1da583794849278 [term 1 FOLLOWER]: Refusing update from remote peer eb84f2b6e8ee42b583570decf843e6c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:08:40.065346 10030 consensus_queue.cc:1035] T ef532e6d7d604ed7bf7c59dcc85025e6 P eb84f2b6e8ee42b583570decf843e6c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "629e64d41cef4647a1da583794849278" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 43881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:08:40.065685 10017 consensus_queue.cc:1035] T ef532e6d7d604ed7bf7c59dcc85025e6 P eb84f2b6e8ee42b583570decf843e6c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "549a11c5d6d24eefa085ac838f8c4845" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 42483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:08:40.076463 10038 mvcc.cc:205] Tried to move back new op lower bound from 7316636139774648320 to 7316636139632840704. Current Snapshot: MvccSnapshot[applied={T|T < 7316636139774648320}]
I20260809 15:08:40.077075 10043 mvcc.cc:205] Tried to move back new op lower bound from 7316636139774648320 to 7316636139632840704. Current Snapshot: MvccSnapshot[applied={T|T < 7316636139774648320}]
I20260809 15:08:40.082289 10042 mvcc.cc:205] Tried to move back new op lower bound from 7316636139774648320 to 7316636139632840704. Current Snapshot: MvccSnapshot[applied={T|T < 7316636139774648320}]
I20260809 15:08:40.091813 9204 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:08:40.092133 9204 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:08:40.096004 9204 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b93e826c76a44f59b380f1dfde5de3fa: Sending DeleteTablet for 3 replicas of tablet ef532e6d7d604ed7bf7c59dcc85025e6
I20260809 15:08:40.097208 9413 tablet_service.cc:1514] Processing DeleteTablet for tablet ef532e6d7d604ed7bf7c59dcc85025e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:08:40 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:45816
I20260809 15:08:40.097251 9870 tablet_service.cc:1514] Processing DeleteTablet for tablet ef532e6d7d604ed7bf7c59dcc85025e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:08:40 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:56848
I20260809 15:08:40.097746 9661 tablet_service.cc:1514] Processing DeleteTablet for tablet ef532e6d7d604ed7bf7c59dcc85025e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:08:40 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:36082
I20260809 15:08:40.104408 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 9315
I20260809 15:08:40.112632 10061 tablet_replica.cc:330] T ef532e6d7d604ed7bf7c59dcc85025e6 P eb84f2b6e8ee42b583570decf843e6c7: stopping tablet replica
I20260809 15:08:40.113320 10061 raft_consensus.cc:2229] T ef532e6d7d604ed7bf7c59dcc85025e6 P eb84f2b6e8ee42b583570decf843e6c7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:08:40.114205 10061 raft_consensus.cc:2258] T ef532e6d7d604ed7bf7c59dcc85025e6 P eb84f2b6e8ee42b583570decf843e6c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:08:40.116178 10057 tablet_replica.cc:330] T ef532e6d7d604ed7bf7c59dcc85025e6 P 549a11c5d6d24eefa085ac838f8c4845: stopping tablet replica
I20260809 15:08:40.116886 10058 tablet_replica.cc:330] T ef532e6d7d604ed7bf7c59dcc85025e6 P 629e64d41cef4647a1da583794849278: stopping tablet replica
I20260809 15:08:40.117342 10058 raft_consensus.cc:2229] T ef532e6d7d604ed7bf7c59dcc85025e6 P 629e64d41cef4647a1da583794849278 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:08:40.117923 10058 raft_consensus.cc:2258] T ef532e6d7d604ed7bf7c59dcc85025e6 P 629e64d41cef4647a1da583794849278 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:08:40.117342 10057 raft_consensus.cc:2229] T ef532e6d7d604ed7bf7c59dcc85025e6 P 549a11c5d6d24eefa085ac838f8c4845 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:08:40.117980 10061 ts_tablet_manager.cc:1905] T ef532e6d7d604ed7bf7c59dcc85025e6 P eb84f2b6e8ee42b583570decf843e6c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:08:40.118734 10057 raft_consensus.cc:2258] T ef532e6d7d604ed7bf7c59dcc85025e6 P 549a11c5d6d24eefa085ac838f8c4845 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:08:40.120275 10058 ts_tablet_manager.cc:1905] T ef532e6d7d604ed7bf7c59dcc85025e6 P 629e64d41cef4647a1da583794849278: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 15:08:40 dist-test-slave-nj21 krb5kdc[9140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.193@KRBTEST.COM
I20260809 15:08:40.123984 10057 ts_tablet_manager.cc:1905] T ef532e6d7d604ed7bf7c59dcc85025e6 P 549a11c5d6d24eefa085ac838f8c4845: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:08:40.126674 10061 ts_tablet_manager.cc:1918] T ef532e6d7d604ed7bf7c59dcc85025e6 P eb84f2b6e8ee42b583570decf843e6c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:08:40.126935 10061 log.cc:1196] T ef532e6d7d604ed7bf7c59dcc85025e6 P eb84f2b6e8ee42b583570decf843e6c7: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-1/wal/wals/ef532e6d7d604ed7bf7c59dcc85025e6
I20260809 15:08:40.127542 10061 ts_tablet_manager.cc:1939] T ef532e6d7d604ed7bf7c59dcc85025e6 P eb84f2b6e8ee42b583570decf843e6c7: Deleting consensus metadata
I20260809 15:08:40.129465 9186 catalog_manager.cc:4841] TS eb84f2b6e8ee42b583570decf843e6c7 (127.24.183.194:39385): tablet ef532e6d7d604ed7bf7c59dcc85025e6 (table test-table [id=aca9430f4bbd498c9bfb47e35adc6b8a]) successfully deleted
I20260809 15:08:40.131390 10005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:40.105808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45822 (local address 127.24.183.193:42483)
0809 15:08:40.106014 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:40.106021 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:40.107979 (+ 1958us) server_negotiation.cc:407] Connection header received
0809 15:08:40.108930 (+ 951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:40.108937 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:40.109027 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:40.109104 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:40.112573 (+ 3469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:40.113812 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:40.116400 (+ 2588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:40.117067 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:40.117196 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:40.123509 (+ 6313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:40.123525 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:40.123543 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:08:40.123589 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:40.127541 (+ 3952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:40.128527 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:40.128542 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:40.128546 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:40.128714 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:40.129193 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:40.129198 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:40.129202 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:40.129741 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:40.130058 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:40.130702 (+ 644us) server_negotiation.cc:299] Negotiation successful
0809 15:08:40.130969 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 15:08:40.131964 10058 ts_tablet_manager.cc:1918] T ef532e6d7d604ed7bf7c59dcc85025e6 P 629e64d41cef4647a1da583794849278: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:08:40.132159 10058 log.cc:1196] T ef532e6d7d604ed7bf7c59dcc85025e6 P 629e64d41cef4647a1da583794849278: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-2/wal/wals/ef532e6d7d604ed7bf7c59dcc85025e6
I20260809 15:08:40.132687 10058 ts_tablet_manager.cc:1939] T ef532e6d7d604ed7bf7c59dcc85025e6 P 629e64d41cef4647a1da583794849278: Deleting consensus metadata
I20260809 15:08:40.134047 10057 ts_tablet_manager.cc:1918] T ef532e6d7d604ed7bf7c59dcc85025e6 P 549a11c5d6d24eefa085ac838f8c4845: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:08:40.134295 10057 log.cc:1196] T ef532e6d7d604ed7bf7c59dcc85025e6 P 549a11c5d6d24eefa085ac838f8c4845: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288021313853-25311-0/minicluster-data/ts-0/wal/wals/ef532e6d7d604ed7bf7c59dcc85025e6
I20260809 15:08:40.134493 9510 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:08:40.134624 9189 catalog_manager.cc:4841] TS 629e64d41cef4647a1da583794849278 (127.24.183.195:43881): tablet ef532e6d7d604ed7bf7c59dcc85025e6 (table test-table [id=aca9430f4bbd498c9bfb47e35adc6b8a]) successfully deleted
I20260809 15:08:40.233670 10057 ts_tablet_manager.cc:1939] T ef532e6d7d604ed7bf7c59dcc85025e6 P 549a11c5d6d24eefa085ac838f8c4845: Deleting consensus metadata
I20260809 15:08:40.234797 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 9315
I20260809 15:08:40.262472 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 9535
W20260809 15:08:40.262491 9188 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.183.193:42483 (error 108)
I20260809 15:08:40.266212 10002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:40.265689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:42483 (local address 127.0.0.1:45832)
0809 15:08:40.265900 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:40.266030 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260809 15:08:40.266490 9188 catalog_manager.cc:4584] TS 549a11c5d6d24eefa085ac838f8c4845 (127.24.183.193:42483): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ef532e6d7d604ed7bf7c59dcc85025e6: Network error: Client connection negotiation failed: client connection to 127.24.183.193:42483: connect: Connection refused (error 111)
Aug 09 15:08:40 dist-test-slave-nj21 krb5kdc[9140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.194@KRBTEST.COM
I20260809 15:08:40.278667 10008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:40.263355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36096 (local address 127.24.183.194:39385)
0809 15:08:40.263555 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:40.263561 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:40.263913 (+ 352us) server_negotiation.cc:407] Connection header received
0809 15:08:40.263999 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:40.264005 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:40.264117 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:40.264207 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:40.265511 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:40.266447 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:40.267899 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:40.268329 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:40.268440 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:40.272505 (+ 4065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:40.272518 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:40.272533 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:08:40.272576 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:40.275709 (+ 3133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:40.276374 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:40.276384 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:40.276387 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:40.276546 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:40.276983 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:40.276989 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:40.276993 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:40.277453 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:40.277815 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:40.278012 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 15:08:40.278253 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 15:08:40.279369 9756 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:08:40.320108 10002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:40.319603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:42483 (local address 127.0.0.1:45838)
0809 15:08:40.319838 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:40.319939 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260809 15:08:40.321112 10002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:40.320754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:42483 (local address 127.0.0.1:45850)
0809 15:08:40.320907 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:40.320991 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 15:08:40.381875 10002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:40.381232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:42483 (local address 127.0.0.1:45856)
0809 15:08:40.381489 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:40.381621 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260809 15:08:40.383036 10002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:40.382624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:42483 (local address 127.0.0.1:45866)
0809 15:08:40.382798 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:40.382892 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 15:08:40.397116 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 9535
I20260809 15:08:40.421767 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 9773
Aug 09 15:08:40 dist-test-slave-nj21 krb5kdc[9140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.195@KRBTEST.COM
I20260809 15:08:40.435926 10003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:40.422545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56862 (local address 127.24.183.195:43881)
0809 15:08:40.422732 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 15:08:40.422738 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:08:40.422759 (+ 21us) server_negotiation.cc:407] Connection header received
0809 15:08:40.422902 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:08:40.422907 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:08:40.422981 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:08:40.423061 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:08:40.424405 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:40.425145 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:40.426356 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:08:40.426786 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:08:40.426909 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:08:40.430953 (+ 4044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:08:40.430967 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:08:40.430984 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:08:40.431019 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:08:40.433670 (+ 2651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:40.434215 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:40.434223 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:40.434226 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:40.434396 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:08:40.434661 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:08:40.434664 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:08:40.434666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:08:40.434988 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:08:40.435212 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 15:08:40.435423 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 15:08:40.435590 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 15:08:40.436551 9966 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:08:40.463398 10002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:40.462882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:42483 (local address 127.0.0.1:45876)
0809 15:08:40.463100 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:40.463207 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 15:08:40.464494 10002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:40.464051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:42483 (local address 127.0.0.1:45884)
0809 15:08:40.464268 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:40.464355 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260809 15:08:40.527869 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 9773
I20260809 15:08:40.551859 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 9160
I20260809 15:08:40.553015 9264 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:08:40.664361 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 9160
I20260809 15:08:40.664456 10002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:08:40.663771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:42483 (local address 127.0.0.1:45890)
0809 15:08:40.664107 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0809 15:08:40.664247 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
2026-08-09T15:08:40Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4376 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:08:40 dist-test-slave-nj21 krb5kdc[10075](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:08:40 dist-test-slave-nj21 krb5kdc[10075](info): set up 2 sockets
Aug 09 15:08:40 dist-test-slave-nj21 krb5kdc[10075](info): commencing operation
krb5kdc: starting...
W20260809 15:08:41.906666 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.002s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:08:42 dist-test-slave-nj21 krb5kdc[10075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288122, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:08:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:08:42Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.24.183.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.24.183.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.24.183.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/rangeradmin_127.24.183.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.24.183.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/rangeradmin_127.24.183.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.24.183.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.24.183.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.24.183.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/rangerlookup_127.24.183.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.24.183.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/rangerlookup_127.24.183.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.183.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.24.183.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.183.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/HTTP_127.24.183.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.183.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/HTTP_127.24.183.212@KRBTEST.COM.keytab.
I20260809 15:08:42.285020 25311 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskXnZFlc/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/postgres -l logfile start
2026-08-09 15:08:44.670 UTC [10123] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 15:08:44.670 UTC [10123] LOG: listening on IPv4 address "127.24.183.212", port 37793
2026-08-09 15:08:44.673 UTC [10123] LOG: listening on Unix socket "/tmp/.s.PGSQL.37793"
2026-08-09 15:08:44.697 UTC [10127] LOG: database system was shut down at 2026-08-09 15:08:43 UTC
2026-08-09 15:08:44.700 UTC [10123] LOG: database system is ready to accept connections
I20260809 15:08:45.793814 25311 mini_postgres.cc:96] Postgres bound to 37793
2026-08-09 15:08:45.819 UTC [10139] FATAL: database "slave" does not exist
127.24.183.212:37793 - accepting connections
I20260809 15:08:45.820564 25311 mini_ranger.cc:161] Starting Ranger...
I20260809 15:08:45.853022 25311 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 15:08:46.294375 25311 mini_ranger.cc:87] Created ranger Postgres database
I20260809 15:08:46.294499 25311 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ranger-admin
2026-08-09 15:08:46,636 [I] DB FLAVOR :POSTGRES
2026-08-09 15:08:46,637 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 15:08:46,637 [I] Checking connection..
2026-08-09 15:08:46,637 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:37793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 15:08:46,883 [I] Checking connection passed.
2026-08-09 15:08:46,883 [I] --------- Verifying version history table ---------
2026-08-09 15:08:46,883 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:37793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 15:08:47,147 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 15:08:47,147 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:37793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 15:08:47,408 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 15:08:47,409 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 15:08:47,409 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:37793/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 15:08:47.652 UTC [10253] WARNING: there is no transaction in progress
2026-08-09 15:08:47,660 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 15:08:47,661 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:37793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 15:08:47,900 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 15:08:47,900 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 15:08:47,900 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:37793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 15:08:48,142 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:37793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 15:08:48,379 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:37793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-nj21.c.gcp-upstream.internal','N') ;"
2026-08-09 15:08:48,622 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:37793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 15:08:48,871 [I] Table x_portal_user does not exist in database ranger
2026-08-09 15:08:48,871 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:37793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 15:08:49,113 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 15:08:49,114 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:37793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 15:08:49,375 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:37793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 15:08:49,634 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 15:08:49,634 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:37793/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 15:08:50.539 UTC [10434] WARNING: there is no transaction in progress
2026-08-09 15:08:50.554 UTC [10434] WARNING: there is no transaction in progress
2026-08-09 15:08:50.569 UTC [10434] WARNING: there is no transaction in progress
2026-08-09 15:08:50.584 UTC [10434] WARNING: there is no transaction in progress
2026-08-09 15:08:50.599 UTC [10434] WARNING: there is no transaction in progress
2026-08-09 15:08:50.618 UTC [10434] WARNING: there is no transaction in progress
2026-08-09 15:08:50.670 UTC [10434] WARNING: there is no transaction in progress
2026-08-09 15:08:50.676 UTC [10434] WARNING: there is no transaction in progress
2026-08-09 15:08:50.688 UTC [10434] WARNING: there is no transaction in progress
2026-08-09 15:08:50.698 UTC [10434] WARNING: there is no transaction in progress
2026-08-09 15:08:50.707 UTC [10434] WARNING: there is no transaction in progress
2026-08-09 15:08:50.721 UTC [10434] WARNING: there is no transaction in progress
2026-08-09 15:08:50.730 UTC [10434] WARNING: there is no transaction in progress
2026-08-09 15:08:50.737 UTC [10434] WARNING: there is no transaction in progress
2026-08-09 15:08:50.743 UTC [10434] WARNING: there is no transaction in progress
2026-08-09 15:08:50.750 UTC [10434] WARNING: there is no transaction in progress
2026-08-09 15:08:51,319 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 15:08:51,319 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:37793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 15:08:51,565 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 15:08:51,566 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:37793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 15:08:51,823 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 15:08:51,823 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:37793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 15:08:52,071 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 15:08:52,071 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:37793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 15:08:52,331 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:37793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 15:08:52,578 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:37793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-nj21.c.gcp-upstream.internal';"
2026-08-09 15:08:52,808 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 15:08:52,809 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:37793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 15:08:53,052 [I] DB_PATCHES have already been applied
I20260809 15:08:53.058882 25311 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ranger-admin:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskXnZFlc/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 15:08:53.058977 25311 mini_ranger.cc:193] Using host: 127.24.183.212
I20260809 15:08:53.081373 25311 mini_ranger.cc:227] Ranger admin URL: http://127.24.183.212:52135
Aug 09, 2026 3:08:53 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 3:08:53 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 3:08:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 3:08:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 3:08:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 3:08:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 3:08:54 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 15:08:54.359184 25311 mini_ranger.cc:160] Time spent starting Ranger: real 8.539s user 0.007s sys 0.059s
Aug 09, 2026 3:09:13 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 3:09:13 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 3:09:13 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 3:09:13 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 3:09:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 3:09:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 3:09:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 3:09:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 3:09:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 3:09:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 3:09:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 3:09:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 3:09:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 3:09:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 3:09:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 3:09:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 3:09:13 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 3:09:14 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 15:09:15.335582 25311 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.24.183.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.24.183.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.24.183.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/rangerkms_127.24.183.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.24.183.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/rangerkms_127.24.183.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.183.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.24.183.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.183.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/HTTP_127.24.183.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.183.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/HTTP_127.24.183.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 15:09:15 dist-test-slave-nj21 krb5kdc[10075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288155, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 15:09:15.529824 25311 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 15:09:15.555990 25311 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 15:09:16.311043 25311 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 15:09:16.311187 25311 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ranger-kms
I20260809 15:09:16.311301 25311 mini_ranger_kms.cc:227] Using postgres at 127.24.183.212:37793
2026-08-09 15:09:16,823 [I] DB FLAVOR :POSTGRES
2026-08-09 15:09:16,823 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 15:09:16,823 [I] Checking connection
2026-08-09 15:09:16,823 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:37793/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 15:09:17,055 [I] connection success
2026-08-09 15:09:17,055 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 15:09:17,055 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 15:09:17,055 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:37793/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 15:09:17,300 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 15:09:17,300 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 15:09:17,300 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 15:09:17,300 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:37793/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 15:09:17,564 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 15:09:18.047760 25311 mini_ranger_kms.cc:310] Created kms service
I20260809 15:09:18.239877 25311 mini_ranger_kms.cc:326] Created kudu user
I20260809 15:09:18.284009 25311 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 15:09:18 dist-test-slave-nj21 krb5kdc[10075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288155, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.24.183.212@KRBTEST.COM
I20260809 15:09:18.712800 25311 mini_ranger_kms.cc:384] Added ranger policy
I20260809 15:09:18.714197 25311 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ranger-kms:/tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 15:09:18.714262 25311 mini_ranger_kms.cc:240] Using host: 127.24.183.212
I20260809 15:09:18.739840 25311 mini_ranger_kms.cc:276] Ranger KMS PID: 10722
I20260809 15:09:18.740051 25311 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.24.183.212:59583
Aug 09, 2026 3:09:19 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 3:09:19 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 3:09:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 3:09:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 3:09:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 3:09:20 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 15:09:20 dist-test-slave-nj21 krb5kdc[10075](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786288160, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.183.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:09:21.276907 25311 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.747s user 0.136s sys 0.415s
I20260809 15:09:21.277330 25311 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 15:09:21.277580 25311 mini_ranger_kms.cc:401] 127.24.183.212:59583/kms/v1/keys
Aug 09 15:09:21 dist-test-slave-nj21 krb5kdc[10075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288160, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.183.212@KRBTEST.COM for HTTP/127.24.183.212@KRBTEST.COM
Aug 09 15:09:21 dist-test-slave-nj21 krb5kdc[10075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288160, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.183.212@KRBTEST.COM for HTTP/127.24.183.212@KRBTEST.COM
Aug 09 15:09:24 dist-test-slave-nj21 krb5kdc[10075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.24.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
I20260809 15:09:24.091905 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:38745
--webserver_interface=127.24.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:39449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:38745
--ranger_config_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.183.212:59583/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:09:24.440064 10810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:09:24.440357 10810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:09:24.440493 10810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:09:24.447585 10810 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:09:24.447721 10810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:09:24.447805 10810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:09:24.447883 10810 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:09:24.447949 10810 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:09:24.457850 10810 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:39449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/master-0/wal
--ranger_kms_url=127.24.183.212:59583/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:38745
--ranger_config_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:38745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:09:24.459098 10810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:09:24.460410 10810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:09:24.472285 10818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:09:24.472255 10816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:09:24.472363 10815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:09:24.472996 10810 server_base.cc:1034] running on GCE node
I20260809 15:09:24.473683 10810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:09:24.474990 10810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:09:24.476183 10810 hybrid_clock.cc:648] HybridClock initialized: now 1786288164476140 us; error 32 us; skew 500 ppm
I20260809 15:09:24.476500 10810 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:09:24 dist-test-slave-nj21 krb5kdc[10075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288164, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:09:24.481840 10810 init.cc:377] Logged in from keytab as kudu/127.24.183.254@KRBTEST.COM (short username kudu)
I20260809 15:09:24.483247 10810 webserver.cc:460] Webserver started at http://127.24.183.254:39767/ using document root <none> and password file <none>
I20260809 15:09:24.483906 10810 fs_manager.cc:359] Metadata directory not provided
I20260809 15:09:24.484017 10810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:09:24.484298 10810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 15:09:24 dist-test-slave-nj21 krb5kdc[10075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288164, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for HTTP/127.24.183.212@KRBTEST.COM
I20260809 15:09:24.535174 10810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "c1256b8a7f38443097164f9a8a6b2484"
format_stamp: "Formatted at 2026-08-09 15:09:24 on dist-test-slave-nj21"
server_key: "6380761ea5480c62666b14921494131a"
server_key_iv: "e517e756ff4bcfca8be4ea0becb436f5"
server_key_version: "kuduclusterkey@0"
I20260809 15:09:24.536087 10810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "c1256b8a7f38443097164f9a8a6b2484"
format_stamp: "Formatted at 2026-08-09 15:09:24 on dist-test-slave-nj21"
server_key: "6380761ea5480c62666b14921494131a"
server_key_iv: "e517e756ff4bcfca8be4ea0becb436f5"
server_key_version: "kuduclusterkey@0"
I20260809 15:09:24.541457 10810 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 15:09:24.561928 10832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:09:24.563361 10810 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:09:24.563544 10810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "c1256b8a7f38443097164f9a8a6b2484"
format_stamp: "Formatted at 2026-08-09 15:09:24 on dist-test-slave-nj21"
server_key: "6380761ea5480c62666b14921494131a"
server_key_iv: "e517e756ff4bcfca8be4ea0becb436f5"
server_key_version: "kuduclusterkey@0"
I20260809 15:09:24.563805 10810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:09:24.590243 10810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:09:24.594230 10810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:09:24.594544 10810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:09:24.613691 10810 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:38745
I20260809 15:09:24.613770 10904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:38745 every 8 connection(s)
I20260809 15:09:24.615006 10810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:09:24.615736 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 10810
I20260809 15:09:24.615932 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:09:24.616793 25311 external_mini_cluster.cc:1428] Setting key 49aa5c348f6226484c413eb83ebe3930
I20260809 15:09:24.620710 10906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 15:09:24 dist-test-slave-nj21 krb5kdc[10075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:09:24.634586 10906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1256b8a7f38443097164f9a8a6b2484: Bootstrap starting.
I20260809 15:09:24.638669 10906 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1256b8a7f38443097164f9a8a6b2484: Neither blocks nor log segments found. Creating new log.
I20260809 15:09:24.639822 10906 log.cc:824] T 00000000000000000000000000000000 P c1256b8a7f38443097164f9a8a6b2484: Log is configured to *not* fsync() on all Append() calls
I20260809 15:09:24.643059 10906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1256b8a7f38443097164f9a8a6b2484: No bootstrap required, opened a new log
I20260809 15:09:24.643718 10910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:09:24.619094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42576 (local address 127.24.183.254:38745)
0809 15:09:24.619987 (+ 893us) server_negotiation.cc:206] Beginning negotiation
0809 15:09:24.620002 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 15:09:24.620051 (+ 49us) server_negotiation.cc:407] Connection header received
0809 15:09:24.622310 (+ 2259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:09:24.622341 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:09:24.622830 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:09:24.623240 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:09:24.624924 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:09:24.627002 (+ 2078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:09:24.628900 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:09:24.629958 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:09:24.630770 (+ 812us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:09:24.635021 (+ 4251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:09:24.635033 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:09:24.635050 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:09:24.635105 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:09:24.638952 (+ 3847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:09:24.639819 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:09:24.639828 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:09:24.639836 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:09:24.640035 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:09:24.640486 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:09:24.640491 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:09:24.640495 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:09:24.641191 (+ 696us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:09:24.641498 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0809 15:09:24.641934 (+ 436us) server_negotiation.cc:299] Negotiation successful
0809 15:09:24.642334 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":397,"threads_started":1}
I20260809 15:09:24.651351 10906 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1256b8a7f38443097164f9a8a6b2484 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1256b8a7f38443097164f9a8a6b2484" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 38745 } }
I20260809 15:09:24.651714 10906 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1256b8a7f38443097164f9a8a6b2484 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:09:24.651810 10906 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1256b8a7f38443097164f9a8a6b2484 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1256b8a7f38443097164f9a8a6b2484, State: Initialized, Role: FOLLOWER
I20260809 15:09:24.652314 10906 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1256b8a7f38443097164f9a8a6b2484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1256b8a7f38443097164f9a8a6b2484" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 38745 } }
I20260809 15:09:24.652529 10906 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1256b8a7f38443097164f9a8a6b2484 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:09:24.652652 10906 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1256b8a7f38443097164f9a8a6b2484 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:09:24.652803 10906 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1256b8a7f38443097164f9a8a6b2484 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:09:24.654911 10906 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1256b8a7f38443097164f9a8a6b2484 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1256b8a7f38443097164f9a8a6b2484" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 38745 } }
I20260809 15:09:24.655355 10906 leader_election.cc:302] T 00000000000000000000000000000000 P c1256b8a7f38443097164f9a8a6b2484 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1256b8a7f38443097164f9a8a6b2484; no voters:
I20260809 15:09:24.655943 10906 leader_election.cc:288] T 00000000000000000000000000000000 P c1256b8a7f38443097164f9a8a6b2484 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:09:24.656073 10913 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1256b8a7f38443097164f9a8a6b2484 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:09:24.656726 10913 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1256b8a7f38443097164f9a8a6b2484 [term 1 LEADER]: Becoming Leader. State: Replica: c1256b8a7f38443097164f9a8a6b2484, State: Running, Role: LEADER
I20260809 15:09:24.657294 10913 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1256b8a7f38443097164f9a8a6b2484 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1256b8a7f38443097164f9a8a6b2484" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 38745 } }
I20260809 15:09:24.657999 10906 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1256b8a7f38443097164f9a8a6b2484 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:09:24.660343 10915 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1256b8a7f38443097164f9a8a6b2484 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1256b8a7f38443097164f9a8a6b2484. Latest consensus state: current_term: 1 leader_uuid: "c1256b8a7f38443097164f9a8a6b2484" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1256b8a7f38443097164f9a8a6b2484" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 38745 } } }
I20260809 15:09:24.660705 10914 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1256b8a7f38443097164f9a8a6b2484 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1256b8a7f38443097164f9a8a6b2484" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1256b8a7f38443097164f9a8a6b2484" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 38745 } } }
I20260809 15:09:24.660976 10915 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1256b8a7f38443097164f9a8a6b2484 [sys.catalog]: This master's current role is: LEADER
I20260809 15:09:24.661032 10914 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1256b8a7f38443097164f9a8a6b2484 [sys.catalog]: This master's current role is: LEADER
I20260809 15:09:24.664649 10906 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 15:09:24.674400 10923 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:09:24.680575 10923 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:09:24.692138 10923 catalog_manager.cc:1347] Generated new cluster ID: 1c6a5745a4ff480f8428c1ce5217557e
I20260809 15:09:24.692258 10923 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:09:24.712637 10923 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:09:24.714738 10923 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:09:24.742775 10923 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1256b8a7f38443097164f9a8a6b2484: Generated new TSK 0
I20260809 15:09:24.743981 10923 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab.
Entry for principal kudu/127.24.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab.
I20260809 15:09:25.517153 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.183.212:59583/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.193:0
--local_ip_for_outbound_sockets=127.24.183.193
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:38745
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:39449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 15:09:25 dist-test-slave-nj21 krb5kdc[10075](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786288165, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 15:09:25.981949 10969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:09:25.982442 10969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:09:25.982666 10969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:09:25.994355 10969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:09:25.994517 10969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:09:25.994760 10969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.193
I20260809 15:09:26.010639 10969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:39449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.24.183.212:59583/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:38745
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.193
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:09:26.012456 10969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:09:26.014186 10969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:09:26.042390 10992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:09:26.042367 10995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:09:26.047885 10969 server_base.cc:1034] running on GCE node
W20260809 15:09:26.048506 10999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:09:26.049368 10969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:09:26.050714 10969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:09:26.052049 10969 hybrid_clock.cc:648] HybridClock initialized: now 1786288166051935 us; error 96 us; skew 500 ppm
I20260809 15:09:26.052465 10969 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:09:26 dist-test-slave-nj21 krb5kdc[10075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288166, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:09:26.056865 10969 init.cc:377] Logged in from keytab as kudu/127.24.183.193@KRBTEST.COM (short username kudu)
I20260809 15:09:26.061275 10969 webserver.cc:460] Webserver started at http://127.24.183.193:34453/ using document root <none> and password file <none>
I20260809 15:09:26.062261 10969 fs_manager.cc:359] Metadata directory not provided
I20260809 15:09:26.062434 10969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:09:26.062883 10969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 15:09:26 dist-test-slave-nj21 krb5kdc[10075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288166, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for HTTP/127.24.183.212@KRBTEST.COM
I20260809 15:09:26.083593 10969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-0/data/instance:
uuid: "ae69fba8b3fd4309855d72a1575005c1"
format_stamp: "Formatted at 2026-08-09 15:09:26 on dist-test-slave-nj21"
server_key: "d78fa590189d56371afb7155c0b0bad4"
server_key_iv: "2ba3a64fc7f98c894c167bd5ad60d8f8"
server_key_version: "kuduclusterkey@0"
I20260809 15:09:26.084824 10969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance:
uuid: "ae69fba8b3fd4309855d72a1575005c1"
format_stamp: "Formatted at 2026-08-09 15:09:26 on dist-test-slave-nj21"
server_key: "d78fa590189d56371afb7155c0b0bad4"
server_key_iv: "2ba3a64fc7f98c894c167bd5ad60d8f8"
server_key_version: "kuduclusterkey@0"
I20260809 15:09:26.098445 10969 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.008s sys 0.003s
I20260809 15:09:26.148355 11009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:09:26.150502 10969 fs_manager.cc:725] Time spent opening block manager: real 0.020s user 0.007s sys 0.012s
I20260809 15:09:26.150835 10969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-0/wal
uuid: "ae69fba8b3fd4309855d72a1575005c1"
format_stamp: "Formatted at 2026-08-09 15:09:26 on dist-test-slave-nj21"
server_key: "d78fa590189d56371afb7155c0b0bad4"
server_key_iv: "2ba3a64fc7f98c894c167bd5ad60d8f8"
server_key_version: "kuduclusterkey@0"
I20260809 15:09:26.151198 10969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:09:26.192607 10969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:09:26.199177 10969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:09:26.199653 10969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:09:26.202978 10969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:09:26.207404 10969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:09:26.207511 10969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:09:26.207649 10969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:09:26.207744 10969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Aug 09 15:09:26 dist-test-slave-nj21 krb5kdc[10075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288165, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for HTTP/127.24.183.212@KRBTEST.COM
I20260809 15:09:26.448536 10969 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.193:45931
I20260809 15:09:26.448586 11176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.193:45931 every 8 connection(s)
I20260809 15:09:26.450420 10969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
I20260809 15:09:26.451071 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 10969
I20260809 15:09:26.451288 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance
I20260809 15:09:26.452211 25311 external_mini_cluster.cc:1428] Setting key fda58fba32b77c1d30d15b7fea9a90fe
Aug 09 15:09:26 dist-test-slave-nj21 krb5kdc[10075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288166, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.183.194@KRBTEST.COM; defaulting to no policy
I20260809 15:09:26.500590 11182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:09:26.454758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:40805 (local address 127.24.183.254:38745)
0809 15:09:26.481124 (+ 26366us) server_negotiation.cc:206] Beginning negotiation
0809 15:09:26.481130 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:09:26.481150 (+ 20us) server_negotiation.cc:407] Connection header received
0809 15:09:26.481209 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:09:26.481214 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:09:26.481306 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:09:26.481413 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:09:26.484514 (+ 3101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:09:26.485613 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:09:26.487903 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:09:26.488509 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:09:26.488596 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:09:26.493024 (+ 4428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:09:26.493031 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:09:26.493035 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:09:26.493085 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:09:26.496442 (+ 3357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:09:26.497445 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:09:26.497451 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:09:26.497455 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:09:26.497787 (+ 332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:09:26.498382 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:09:26.498388 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:09:26.498392 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:09:26.498755 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:09:26.498891 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 15:09:26.499949 (+ 1058us) server_negotiation.cc:299] Negotiation successful
0809 15:09:26.500180 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26239,"thread_start_us":27568,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.194@KRBTEST.COM" created.
I20260809 15:09:26.502868 11181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:09:26.454179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:38745 (local address 127.24.183.193:40805)
0809 15:09:26.477607 (+ 23428us) negotiation.cc:107] Waiting for socket to connect
0809 15:09:26.477675 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0809 15:09:26.480343 (+ 2668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:09:26.482726 (+ 2383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:09:26.482738 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:09:26.483370 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:09:26.484307 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:09:26.484339 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:09:26.485823 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:09:26.485828 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:09:26.487746 (+ 1918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:09:26.487760 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:09:26.488930 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:09:26.488938 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:09:26.489206 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:09:26.490012 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:09:26.490051 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:09:26.492763 (+ 2712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:09:26.496696 (+ 3933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:09:26.496704 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:09:26.496722 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:09:26.497296 (+ 574us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:09:26.497946 (+ 650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:09:26.497950 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:09:26.497954 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:09:26.498223 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:09:26.498987 (+ 764us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:09:26.498995 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:09:26.499455 (+ 460us) client_negotiation.cc:769] Sending connection context
0809 15:09:26.500930 (+ 1475us) client_negotiation.cc:240] Negotiation successful
0809 15:09:26.501524 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23022,"thread_start_us":27212,"threads_started":1}
I20260809 15:09:26.504944 11177 heartbeater.cc:347] Connected to a master server at 127.24.183.254:38745
I20260809 15:09:26.505261 11177 heartbeater.cc:464] Registering TS with master...
I20260809 15:09:26.506081 11177 heartbeater.cc:511] Master 127.24.183.254:38745 requested a full tablet report, sending...
I20260809 15:09:26.508596 10859 ts_manager.cc:194] Registered new tserver with Master: ae69fba8b3fd4309855d72a1575005c1 (127.24.183.193:45931)
I20260809 15:09:26.510573 10859 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.193@KRBTEST.COM'} at 127.24.183.193:40805
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab.
Entry for principal kudu/127.24.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab.
I20260809 15:09:26.553115 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.183.212:59583/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.194:0
--local_ip_for_outbound_sockets=127.24.183.194
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:38745
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:39449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 15:09:26 dist-test-slave-nj21 krb5kdc[10075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288165, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for HTTP/127.24.183.212@KRBTEST.COM
W20260809 15:09:26.984449 11187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:09:26.984726 11187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:09:26.984862 11187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:09:26.992110 11187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:09:26.992225 11187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:09:26.992372 11187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.194
I20260809 15:09:27.002102 11187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:39449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.24.183.212:59583/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:38745
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.194
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:09:27.003418 11187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:09:27.004779 11187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:09:27.018365 11213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:09:27.018572 11210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:09:27.019090 11209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:09:27.019446 11187 server_base.cc:1034] running on GCE node
I20260809 15:09:27.020145 11187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:09:27.021204 11187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:09:27.022425 11187 hybrid_clock.cc:648] HybridClock initialized: now 1786288167022361 us; error 56 us; skew 500 ppm
I20260809 15:09:27.022751 11187 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:09:27 dist-test-slave-nj21 krb5kdc[10075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288167, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:09:27.025974 11187 init.cc:377] Logged in from keytab as kudu/127.24.183.194@KRBTEST.COM (short username kudu)
I20260809 15:09:27.027426 11187 webserver.cc:460] Webserver started at http://127.24.183.194:41437/ using document root <none> and password file <none>
I20260809 15:09:27.028110 11187 fs_manager.cc:359] Metadata directory not provided
I20260809 15:09:27.028219 11187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:09:27.028553 11187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 15:09:27 dist-test-slave-nj21 krb5kdc[10075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288167, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for HTTP/127.24.183.212@KRBTEST.COM
I20260809 15:09:27.045337 11187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-1/data/instance:
uuid: "e7aae91f56ad4fa1b1fcca3d3c9e01ed"
format_stamp: "Formatted at 2026-08-09 15:09:27 on dist-test-slave-nj21"
server_key: "6eed763c04bb278ed7f1296350b69793"
server_key_iv: "4ee1951879eca995f9ec797dc3629756"
server_key_version: "kuduclusterkey@0"
I20260809 15:09:27.046152 11187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance:
uuid: "e7aae91f56ad4fa1b1fcca3d3c9e01ed"
format_stamp: "Formatted at 2026-08-09 15:09:27 on dist-test-slave-nj21"
server_key: "6eed763c04bb278ed7f1296350b69793"
server_key_iv: "4ee1951879eca995f9ec797dc3629756"
server_key_version: "kuduclusterkey@0"
I20260809 15:09:27.051558 11187 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260809 15:09:27.080381 11223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:09:27.082046 11187 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.004s sys 0.008s
I20260809 15:09:27.082230 11187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-1/wal
uuid: "e7aae91f56ad4fa1b1fcca3d3c9e01ed"
format_stamp: "Formatted at 2026-08-09 15:09:27 on dist-test-slave-nj21"
server_key: "6eed763c04bb278ed7f1296350b69793"
server_key_iv: "4ee1951879eca995f9ec797dc3629756"
server_key_version: "kuduclusterkey@0"
I20260809 15:09:27.082473 11187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:09:27.102396 11187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:09:27.106341 11187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:09:27.106643 11187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:09:27.107651 11187 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:09:27.109191 11187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:09:27.109270 11187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:09:27.109429 11187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:09:27.109499 11187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:09:27.149362 11187 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.194:38443
I20260809 15:09:27.149379 11404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.194:38443 every 8 connection(s)
I20260809 15:09:27.150786 11187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
I20260809 15:09:27.161103 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 11187
I20260809 15:09:27.161293 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance
I20260809 15:09:27.161840 25311 external_mini_cluster.cc:1428] Setting key 44c75c162e910da4fddb03497a9cbdb9
Aug 09 15:09:27 dist-test-slave-nj21 krb5kdc[10075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288167, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:09:27.177863 11409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:09:27.153621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:48993 (local address 127.24.183.254:38745)
0809 15:09:27.154152 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0809 15:09:27.154158 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:09:27.155137 (+ 979us) server_negotiation.cc:407] Connection header received
0809 15:09:27.157095 (+ 1958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:09:27.157102 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:09:27.157207 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:09:27.157346 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:09:27.158873 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:09:27.160147 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:09:27.162257 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:09:27.162985 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:09:27.163117 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:09:27.171064 (+ 7947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:09:27.171071 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:09:27.171075 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:09:27.171114 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:09:27.174439 (+ 3325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:09:27.175420 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:09:27.175426 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:09:27.175430 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:09:27.175568 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:09:27.176055 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:09:27.176060 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:09:27.176064 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:09:27.176356 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:09:27.176496 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 15:09:27.177235 (+ 739us) server_negotiation.cc:299] Negotiation successful
0809 15:09:27.177475 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":262,"threads_started":1}
I20260809 15:09:27.178887 11410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:09:27.154170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:38745 (local address 127.24.183.194:48993)
0809 15:09:27.154967 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0809 15:09:27.155013 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 15:09:27.156796 (+ 1783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:09:27.157521 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:09:27.157529 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:09:27.158079 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:09:27.158702 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:09:27.158722 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:09:27.160296 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:09:27.160300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:09:27.162088 (+ 1788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:09:27.162104 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:09:27.163217 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:09:27.163224 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:09:27.163512 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:09:27.167929 (+ 4417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:09:27.167964 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:09:27.170790 (+ 2826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:09:27.174596 (+ 3806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:09:27.174606 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:09:27.174625 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:09:27.175278 (+ 653us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:09:27.175702 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:09:27.175708 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:09:27.175712 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:09:27.175937 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:09:27.176493 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:09:27.176501 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:09:27.176839 (+ 338us) client_negotiation.cc:769] Sending connection context
0809 15:09:27.177092 (+ 253us) client_negotiation.cc:240] Negotiation successful
0809 15:09:27.177622 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":334,"threads_started":1}
I20260809 15:09:27.180335 11405 heartbeater.cc:347] Connected to a master server at 127.24.183.254:38745
I20260809 15:09:27.180649 11405 heartbeater.cc:464] Registering TS with master...
I20260809 15:09:27.181362 11405 heartbeater.cc:511] Master 127.24.183.254:38745 requested a full tablet report, sending...
I20260809 15:09:27.182767 10859 ts_manager.cc:194] Registered new tserver with Master: e7aae91f56ad4fa1b1fcca3d3c9e01ed (127.24.183.194:38443)
I20260809 15:09:27.183581 10859 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.194@KRBTEST.COM'} at 127.24.183.194:48993
WARNING: no policy specified for kudu/127.24.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab.
Entry for principal kudu/127.24.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab.
I20260809 15:09:27.232692 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.183.212:59583/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.195:0
--local_ip_for_outbound_sockets=127.24.183.195
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:38745
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:39449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:09:27.514252 11177 heartbeater.cc:503] Master 127.24.183.254:38745 was elected leader, sending a full tablet report...
W20260809 15:09:27.572031 11415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:09:27.572292 11415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:09:27.572397 11415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:09:27.579455 11415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:09:27.579564 11415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:09:27.579708 11415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.195
I20260809 15:09:27.589355 11415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:39449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.24.183.212:59583/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:38745
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.195
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:09:27.590684 11415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:09:27.591961 11415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:09:27.604632 11423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:09:27.604694 11425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:09:27.604660 11421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:09:27.605733 11415 server_base.cc:1034] running on GCE node
I20260809 15:09:27.606276 11415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:09:27.607172 11415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:09:27.608373 11415 hybrid_clock.cc:648] HybridClock initialized: now 1786288167608312 us; error 45 us; skew 500 ppm
I20260809 15:09:27.608676 11415 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:09:27 dist-test-slave-nj21 krb5kdc[10075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288167, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:09:27.611616 11415 init.cc:377] Logged in from keytab as kudu/127.24.183.195@KRBTEST.COM (short username kudu)
I20260809 15:09:27.613023 11415 webserver.cc:460] Webserver started at http://127.24.183.195:41801/ using document root <none> and password file <none>
I20260809 15:09:27.613780 11415 fs_manager.cc:359] Metadata directory not provided
I20260809 15:09:27.613896 11415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:09:27.614189 11415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 15:09:27 dist-test-slave-nj21 krb5kdc[10075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288167, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for HTTP/127.24.183.212@KRBTEST.COM
I20260809 15:09:27.631072 11415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-2/data/instance:
uuid: "16609e7c15b14fa7be968382cdc14db1"
format_stamp: "Formatted at 2026-08-09 15:09:27 on dist-test-slave-nj21"
server_key: "9c92d2de149aaf8fd60a34dda3b9096a"
server_key_iv: "c56369d1c50dac4ab37de4e35343314e"
server_key_version: "kuduclusterkey@0"
I20260809 15:09:27.632087 11415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance:
uuid: "16609e7c15b14fa7be968382cdc14db1"
format_stamp: "Formatted at 2026-08-09 15:09:27 on dist-test-slave-nj21"
server_key: "9c92d2de149aaf8fd60a34dda3b9096a"
server_key_iv: "c56369d1c50dac4ab37de4e35343314e"
server_key_version: "kuduclusterkey@0"
I20260809 15:09:27.638347 11415 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260809 15:09:27.654937 11441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:09:27.656363 11415 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:09:27.656510 11415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-2/wal
uuid: "16609e7c15b14fa7be968382cdc14db1"
format_stamp: "Formatted at 2026-08-09 15:09:27 on dist-test-slave-nj21"
server_key: "9c92d2de149aaf8fd60a34dda3b9096a"
server_key_iv: "c56369d1c50dac4ab37de4e35343314e"
server_key_version: "kuduclusterkey@0"
I20260809 15:09:27.656700 11415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:09:27.687498 11415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:09:27.691157 11415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:09:27.691416 11415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:09:27.692381 11415 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:09:27.693735 11415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:09:27.693810 11415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:09:27.693915 11415 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:09:27.693961 11415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:09:27.725703 11415 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.195:46781
I20260809 15:09:27.725736 11609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.195:46781 every 8 connection(s)
I20260809 15:09:27.727021 11415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
I20260809 15:09:27.729671 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 11415
I20260809 15:09:27.729835 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance
I20260809 15:09:27.730276 25311 external_mini_cluster.cc:1428] Setting key b6b8f8f43eb085a5fc201ef789932340
Aug 09 15:09:27 dist-test-slave-nj21 krb5kdc[10075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288167, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:09:27.746939 11614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:09:27.729641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:43385 (local address 127.24.183.254:38745)
0809 15:09:27.732080 (+ 2439us) server_negotiation.cc:206] Beginning negotiation
0809 15:09:27.732084 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:09:27.732103 (+ 19us) server_negotiation.cc:407] Connection header received
0809 15:09:27.733581 (+ 1478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:09:27.733585 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:09:27.733652 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:09:27.733730 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:09:27.735123 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:09:27.735894 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:09:27.737411 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:09:27.737847 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:09:27.737951 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:09:27.741281 (+ 3330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:09:27.741287 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:09:27.741290 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:09:27.741333 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:09:27.743781 (+ 2448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:09:27.744589 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:09:27.744594 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:09:27.744597 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:09:27.744771 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:09:27.745197 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:09:27.745203 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:09:27.745206 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:09:27.745489 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:09:27.745639 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 15:09:27.746345 (+ 706us) server_negotiation.cc:299] Negotiation successful
0809 15:09:27.746576 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2347,"thread_start_us":2182,"threads_started":1}
I20260809 15:09:27.747954 11615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:09:27.730243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:38745 (local address 127.24.183.195:43385)
0809 15:09:27.731250 (+ 1007us) negotiation.cc:107] Waiting for socket to connect
0809 15:09:27.731302 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 15:09:27.733301 (+ 1999us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:09:27.733953 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:09:27.733960 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:09:27.734385 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:09:27.734965 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:09:27.734983 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:09:27.736046 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:09:27.736050 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:09:27.737265 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:09:27.737278 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:09:27.738034 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:09:27.738039 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:09:27.738241 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:09:27.738860 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:09:27.738883 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:09:27.741027 (+ 2144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:09:27.743921 (+ 2894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:09:27.743931 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:09:27.743947 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:09:27.744477 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:09:27.744880 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:09:27.744886 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:09:27.744889 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:09:27.745095 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:09:27.745623 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:09:27.745630 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:09:27.745982 (+ 352us) client_negotiation.cc:769] Sending connection context
0809 15:09:27.746244 (+ 262us) client_negotiation.cc:240] Negotiation successful
0809 15:09:27.746781 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"thread_start_us":422,"threads_started":1}
I20260809 15:09:27.749302 11610 heartbeater.cc:347] Connected to a master server at 127.24.183.254:38745
I20260809 15:09:27.749624 11610 heartbeater.cc:464] Registering TS with master...
I20260809 15:09:27.750332 11610 heartbeater.cc:511] Master 127.24.183.254:38745 requested a full tablet report, sending...
I20260809 15:09:27.751754 10859 ts_manager.cc:194] Registered new tserver with Master: 16609e7c15b14fa7be968382cdc14db1 (127.24.183.195:46781)
I20260809 15:09:27.752547 10859 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.195@KRBTEST.COM'} at 127.24.183.195:43385
I20260809 15:09:27.758061 25311 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:09:27.774689 11614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:09:27.762886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44132 (local address 127.24.183.254:38745)
0809 15:09:27.763076 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 15:09:27.763083 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:09:27.763495 (+ 412us) server_negotiation.cc:407] Connection header received
0809 15:09:27.763639 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:09:27.763642 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:09:27.763763 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:09:27.763841 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:09:27.765186 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:09:27.765904 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:09:27.767263 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:09:27.767703 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:09:27.767808 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:09:27.769923 (+ 2115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:09:27.769928 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:09:27.769932 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:09:27.769961 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:09:27.772447 (+ 2486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:09:27.773095 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:09:27.773098 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:09:27.773100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:09:27.773210 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:09:27.773540 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:09:27.773543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:09:27.773545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:09:27.773791 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:09:27.773957 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 15:09:27.774256 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 15:09:27.774426 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 15:09:27.778648 10859 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44132:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:09:27.785761 10859 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:09:27.808797 11632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:09:27.799144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42548 (local address 127.24.183.193:45931)
0809 15:09:27.801364 (+ 2220us) server_negotiation.cc:206] Beginning negotiation
0809 15:09:27.801373 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:09:27.801403 (+ 30us) server_negotiation.cc:407] Connection header received
0809 15:09:27.801511 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:09:27.801517 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:09:27.801734 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:09:27.801929 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:09:27.802811 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:09:27.804250 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:09:27.806660 (+ 2410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:09:27.807616 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:09:27.807856 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:09:27.808043 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 15:09:27.808300 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 15:09:27.808477 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2047,"thread_start_us":2678,"threads_started":1}
I20260809 15:09:27.808869 11631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:09:27.798821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:45931 (local address 127.0.0.1:42548)
0809 15:09:27.800716 (+ 1895us) negotiation.cc:107] Waiting for socket to connect
0809 15:09:27.800734 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:09:27.800841 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:09:27.802174 (+ 1333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:09:27.802179 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:09:27.802205 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:09:27.802628 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:09:27.802640 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:09:27.804366 (+ 1726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:09:27.804370 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:09:27.806526 (+ 2156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:09:27.806536 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:09:27.807843 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:09:27.807857 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:09:27.808044 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:09:27.808112 (+ 68us) client_negotiation.cc:769] Sending connection context
0809 15:09:27.808235 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 15:09:27.808411 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1784,"mutex_wait_us":2,"thread_start_us":1651,"threads_started":1}
I20260809 15:09:27.808997 11628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:09:27.797821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:38443 (local address 127.0.0.1:39934)
0809 15:09:27.798844 (+ 1023us) negotiation.cc:107] Waiting for socket to connect
0809 15:09:27.798882 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 15:09:27.799063 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:09:27.801024 (+ 1961us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:09:27.801031 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:09:27.801068 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:09:27.801530 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:09:27.801544 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:09:27.803708 (+ 2164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:09:27.803714 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:09:27.806342 (+ 2628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:09:27.806368 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:09:27.808174 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:09:27.808181 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:09:27.808348 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:09:27.808382 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 15:09:27.808463 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 15:09:27.808588 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"mutex_wait_us":60,"thread_start_us":300,"threads_started":1}
I20260809 15:09:27.809949 11629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:09:27.797941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39934 (local address 127.24.183.194:38443)
0809 15:09:27.800378 (+ 2437us) server_negotiation.cc:206] Beginning negotiation
0809 15:09:27.800388 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:09:27.800431 (+ 43us) server_negotiation.cc:407] Connection header received
0809 15:09:27.800532 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:09:27.800539 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:09:27.800796 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:09:27.801757 (+ 961us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:09:27.802016 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:09:27.803538 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:09:27.806519 (+ 2981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:09:27.807953 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:09:27.809307 (+ 1354us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:09:27.809498 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 15:09:27.809554 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 15:09:27.809672 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2288,"thread_start_us":2183,"threads_started":1}
I20260809 15:09:27.811190 11639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:09:27.801479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:46781 (local address 127.0.0.1:53260)
0809 15:09:27.802461 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0809 15:09:27.802483 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 15:09:27.802621 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:09:27.803586 (+ 965us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:09:27.803590 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:09:27.803612 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:09:27.804119 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:09:27.804131 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:09:27.805883 (+ 1752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:09:27.805889 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:09:27.807801 (+ 1912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:09:27.807816 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:09:27.810442 (+ 2626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:09:27.810448 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:09:27.810612 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:09:27.810641 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 15:09:27.810724 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 15:09:27.810840 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":834,"mutex_wait_us":6,"thread_start_us":698,"threads_started":1}
I20260809 15:09:27.811288 11633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:09:27.801128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53260 (local address 127.24.183.195:46781)
0809 15:09:27.803166 (+ 2038us) server_negotiation.cc:206] Beginning negotiation
0809 15:09:27.803172 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:09:27.803196 (+ 24us) server_negotiation.cc:407] Connection header received
0809 15:09:27.803271 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:09:27.803275 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:09:27.803424 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:09:27.803588 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:09:27.804328 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:09:27.805752 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:09:27.807994 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:09:27.810251 (+ 2257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:09:27.810438 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:09:27.810602 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 15:09:27.810801 (+ 199us) server_negotiation.cc:299] Negotiation successful
0809 15:09:27.810969 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1908,"thread_start_us":1755,"threads_started":1}
I20260809 15:09:27.813392 11306 tablet_service.cc:1467] Processing CreateTablet for tablet 1744600000b94366a308f5597aed4a6f (DEFAULT_TABLE table=test-table [id=05160d2ec85c4f399d6ece1a7f87bcb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:09:27.813414 11082 tablet_service.cc:1467] Processing CreateTablet for tablet 1744600000b94366a308f5597aed4a6f (DEFAULT_TABLE table=test-table [id=05160d2ec85c4f399d6ece1a7f87bcb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:09:27.814530 11517 tablet_service.cc:1467] Processing CreateTablet for tablet 1744600000b94366a308f5597aed4a6f (DEFAULT_TABLE table=test-table [id=05160d2ec85c4f399d6ece1a7f87bcb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:09:27.815012 11306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1744600000b94366a308f5597aed4a6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:09:27.815460 11082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1744600000b94366a308f5597aed4a6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:09:27.815707 11517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1744600000b94366a308f5597aed4a6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:09:27.828903 11640 tablet_bootstrap.cc:492] T 1744600000b94366a308f5597aed4a6f P 16609e7c15b14fa7be968382cdc14db1: Bootstrap starting.
I20260809 15:09:27.832144 11643 tablet_bootstrap.cc:492] T 1744600000b94366a308f5597aed4a6f P ae69fba8b3fd4309855d72a1575005c1: Bootstrap starting.
I20260809 15:09:27.832237 11640 tablet_bootstrap.cc:654] T 1744600000b94366a308f5597aed4a6f P 16609e7c15b14fa7be968382cdc14db1: Neither blocks nor log segments found. Creating new log.
I20260809 15:09:27.833544 11641 tablet_bootstrap.cc:492] T 1744600000b94366a308f5597aed4a6f P e7aae91f56ad4fa1b1fcca3d3c9e01ed: Bootstrap starting.
I20260809 15:09:27.833586 11640 log.cc:824] T 1744600000b94366a308f5597aed4a6f P 16609e7c15b14fa7be968382cdc14db1: Log is configured to *not* fsync() on all Append() calls
I20260809 15:09:27.835807 11643 tablet_bootstrap.cc:654] T 1744600000b94366a308f5597aed4a6f P ae69fba8b3fd4309855d72a1575005c1: Neither blocks nor log segments found. Creating new log.
I20260809 15:09:27.836964 11641 tablet_bootstrap.cc:654] T 1744600000b94366a308f5597aed4a6f P e7aae91f56ad4fa1b1fcca3d3c9e01ed: Neither blocks nor log segments found. Creating new log.
I20260809 15:09:27.837033 11643 log.cc:824] T 1744600000b94366a308f5597aed4a6f P ae69fba8b3fd4309855d72a1575005c1: Log is configured to *not* fsync() on all Append() calls
I20260809 15:09:27.837083 11640 tablet_bootstrap.cc:492] T 1744600000b94366a308f5597aed4a6f P 16609e7c15b14fa7be968382cdc14db1: No bootstrap required, opened a new log
I20260809 15:09:27.837352 11640 ts_tablet_manager.cc:1397] T 1744600000b94366a308f5597aed4a6f P 16609e7c15b14fa7be968382cdc14db1: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 15:09:27.838256 11641 log.cc:824] T 1744600000b94366a308f5597aed4a6f P e7aae91f56ad4fa1b1fcca3d3c9e01ed: Log is configured to *not* fsync() on all Append() calls
I20260809 15:09:27.839856 11643 tablet_bootstrap.cc:492] T 1744600000b94366a308f5597aed4a6f P ae69fba8b3fd4309855d72a1575005c1: No bootstrap required, opened a new log
I20260809 15:09:27.840088 11643 ts_tablet_manager.cc:1397] T 1744600000b94366a308f5597aed4a6f P ae69fba8b3fd4309855d72a1575005c1: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20260809 15:09:27.841207 11641 tablet_bootstrap.cc:492] T 1744600000b94366a308f5597aed4a6f P e7aae91f56ad4fa1b1fcca3d3c9e01ed: No bootstrap required, opened a new log
I20260809 15:09:27.841459 11641 ts_tablet_manager.cc:1397] T 1744600000b94366a308f5597aed4a6f P e7aae91f56ad4fa1b1fcca3d3c9e01ed: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 15:09:27.847198 11643 raft_consensus.cc:348] T 1744600000b94366a308f5597aed4a6f P ae69fba8b3fd4309855d72a1575005c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7aae91f56ad4fa1b1fcca3d3c9e01ed" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 38443 } } peers { permanent_uuid: "ae69fba8b3fd4309855d72a1575005c1" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 45931 } } peers { permanent_uuid: "16609e7c15b14fa7be968382cdc14db1" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 46781 } }
I20260809 15:09:27.847266 11640 raft_consensus.cc:348] T 1744600000b94366a308f5597aed4a6f P 16609e7c15b14fa7be968382cdc14db1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7aae91f56ad4fa1b1fcca3d3c9e01ed" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 38443 } } peers { permanent_uuid: "ae69fba8b3fd4309855d72a1575005c1" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 45931 } } peers { permanent_uuid: "16609e7c15b14fa7be968382cdc14db1" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 46781 } }
I20260809 15:09:27.847549 11643 raft_consensus.cc:374] T 1744600000b94366a308f5597aed4a6f P ae69fba8b3fd4309855d72a1575005c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:09:27.847641 11643 raft_consensus.cc:729] T 1744600000b94366a308f5597aed4a6f P ae69fba8b3fd4309855d72a1575005c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae69fba8b3fd4309855d72a1575005c1, State: Initialized, Role: FOLLOWER
I20260809 15:09:27.847741 11640 raft_consensus.cc:374] T 1744600000b94366a308f5597aed4a6f P 16609e7c15b14fa7be968382cdc14db1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:09:27.847851 11640 raft_consensus.cc:729] T 1744600000b94366a308f5597aed4a6f P 16609e7c15b14fa7be968382cdc14db1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16609e7c15b14fa7be968382cdc14db1, State: Initialized, Role: FOLLOWER
I20260809 15:09:27.848282 11643 consensus_queue.cc:260] T 1744600000b94366a308f5597aed4a6f P ae69fba8b3fd4309855d72a1575005c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7aae91f56ad4fa1b1fcca3d3c9e01ed" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 38443 } } peers { permanent_uuid: "ae69fba8b3fd4309855d72a1575005c1" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 45931 } } peers { permanent_uuid: "16609e7c15b14fa7be968382cdc14db1" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 46781 } }
I20260809 15:09:27.848498 11640 consensus_queue.cc:260] T 1744600000b94366a308f5597aed4a6f P 16609e7c15b14fa7be968382cdc14db1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7aae91f56ad4fa1b1fcca3d3c9e01ed" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 38443 } } peers { permanent_uuid: "ae69fba8b3fd4309855d72a1575005c1" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 45931 } } peers { permanent_uuid: "16609e7c15b14fa7be968382cdc14db1" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 46781 } }
I20260809 15:09:27.849747 11610 heartbeater.cc:503] Master 127.24.183.254:38745 was elected leader, sending a full tablet report...
I20260809 15:09:27.850323 11640 ts_tablet_manager.cc:1428] T 1744600000b94366a308f5597aed4a6f P 16609e7c15b14fa7be968382cdc14db1: Time spent starting tablet: real 0.013s user 0.005s sys 0.008s
I20260809 15:09:27.851354 11641 raft_consensus.cc:348] T 1744600000b94366a308f5597aed4a6f P e7aae91f56ad4fa1b1fcca3d3c9e01ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7aae91f56ad4fa1b1fcca3d3c9e01ed" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 38443 } } peers { permanent_uuid: "ae69fba8b3fd4309855d72a1575005c1" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 45931 } } peers { permanent_uuid: "16609e7c15b14fa7be968382cdc14db1" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 46781 } }
I20260809 15:09:27.851764 11641 raft_consensus.cc:374] T 1744600000b94366a308f5597aed4a6f P e7aae91f56ad4fa1b1fcca3d3c9e01ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:09:27.851863 11641 raft_consensus.cc:729] T 1744600000b94366a308f5597aed4a6f P e7aae91f56ad4fa1b1fcca3d3c9e01ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7aae91f56ad4fa1b1fcca3d3c9e01ed, State: Initialized, Role: FOLLOWER
I20260809 15:09:27.852429 11641 consensus_queue.cc:260] T 1744600000b94366a308f5597aed4a6f P e7aae91f56ad4fa1b1fcca3d3c9e01ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7aae91f56ad4fa1b1fcca3d3c9e01ed" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 38443 } } peers { permanent_uuid: "ae69fba8b3fd4309855d72a1575005c1" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 45931 } } peers { permanent_uuid: "16609e7c15b14fa7be968382cdc14db1" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 46781 } }
I20260809 15:09:27.853098 11643 ts_tablet_manager.cc:1428] T 1744600000b94366a308f5597aed4a6f P ae69fba8b3fd4309855d72a1575005c1: Time spent starting tablet: real 0.013s user 0.003s sys 0.008s
I20260809 15:09:27.854163 11405 heartbeater.cc:503] Master 127.24.183.254:38745 was elected leader, sending a full tablet report...
I20260809 15:09:27.856053 11641 ts_tablet_manager.cc:1428] T 1744600000b94366a308f5597aed4a6f P e7aae91f56ad4fa1b1fcca3d3c9e01ed: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
W20260809 15:09:27.902788 11407 tablet.cc:2369] T 1744600000b94366a308f5597aed4a6f P e7aae91f56ad4fa1b1fcca3d3c9e01ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:09:27.960219 11180 tablet.cc:2369] T 1744600000b94366a308f5597aed4a6f P ae69fba8b3fd4309855d72a1575005c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:09:27.978677 11613 tablet.cc:2369] T 1744600000b94366a308f5597aed4a6f P 16609e7c15b14fa7be968382cdc14db1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:09:28.182626 11650 raft_consensus.cc:482] T 1744600000b94366a308f5597aed4a6f P 16609e7c15b14fa7be968382cdc14db1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:09:28.182946 11650 raft_consensus.cc:504] T 1744600000b94366a308f5597aed4a6f P 16609e7c15b14fa7be968382cdc14db1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7aae91f56ad4fa1b1fcca3d3c9e01ed" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 38443 } } peers { permanent_uuid: "ae69fba8b3fd4309855d72a1575005c1" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 45931 } } peers { permanent_uuid: "16609e7c15b14fa7be968382cdc14db1" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 46781 } }
I20260809 15:09:28.184705 11650 leader_election.cc:288] T 1744600000b94366a308f5597aed4a6f P 16609e7c15b14fa7be968382cdc14db1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7aae91f56ad4fa1b1fcca3d3c9e01ed (127.24.183.194:38443), ae69fba8b3fd4309855d72a1575005c1 (127.24.183.193:45931)
I20260809 15:09:28.191175 11629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:09:28.185168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:39881 (local address 127.24.183.194:38443)
0809 15:09:28.185347 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 15:09:28.185351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:09:28.185415 (+ 64us) server_negotiation.cc:407] Connection header received
0809 15:09:28.185667 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:09:28.185672 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:09:28.185754 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:09:28.185855 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:09:28.186506 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:09:28.187492 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:09:28.189357 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:09:28.190379 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:09:28.190491 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:09:28.190545 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 15:09:28.190871 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 15:09:28.190957 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 15:09:28.191264 11615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:09:28.185117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:38443 (local address 127.24.183.195:39881)
0809 15:09:28.185317 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 15:09:28.185338 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 15:09:28.185482 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:09:28.186106 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:09:28.186110 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:09:28.186137 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:09:28.186352 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:09:28.186364 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:09:28.187630 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:09:28.187636 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:09:28.189190 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:09:28.189203 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:09:28.190535 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:09:28.190541 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:09:28.190720 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:09:28.190757 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 15:09:28.190856 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 15:09:28.191000 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 15:09:28.191901 11632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:09:28.185755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:54153 (local address 127.24.183.193:45931)
0809 15:09:28.186033 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0809 15:09:28.186038 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:09:28.186202 (+ 164us) server_negotiation.cc:407] Connection header received
0809 15:09:28.186369 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:09:28.186374 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:09:28.186465 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:09:28.186542 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:09:28.187149 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:09:28.188092 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:09:28.190035 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:09:28.191014 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:09:28.191156 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:09:28.191220 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 15:09:28.191590 (+ 370us) server_negotiation.cc:299] Negotiation successful
0809 15:09:28.191706 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260809 15:09:28.192028 11653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:09:28.185513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:45931 (local address 127.24.183.195:54153)
0809 15:09:28.186113 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0809 15:09:28.186128 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:09:28.186248 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:09:28.186590 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:09:28.186595 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:09:28.186621 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:09:28.186992 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:09:28.187005 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:09:28.188249 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:09:28.188253 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:09:28.189883 (+ 1630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:09:28.189898 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:09:28.191226 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:09:28.191233 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:09:28.191416 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:09:28.191450 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 15:09:28.191570 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 15:09:28.191715 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":313,"threads_started":1}
I20260809 15:09:28.192425 11338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1744600000b94366a308f5597aed4a6f" candidate_uuid: "16609e7c15b14fa7be968382cdc14db1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7aae91f56ad4fa1b1fcca3d3c9e01ed" is_pre_election: true
I20260809 15:09:28.192953 11338 raft_consensus.cc:2454] T 1744600000b94366a308f5597aed4a6f P e7aae91f56ad4fa1b1fcca3d3c9e01ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16609e7c15b14fa7be968382cdc14db1 in term 0.
I20260809 15:09:28.193102 11109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1744600000b94366a308f5597aed4a6f" candidate_uuid: "16609e7c15b14fa7be968382cdc14db1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae69fba8b3fd4309855d72a1575005c1" is_pre_election: true
I20260809 15:09:28.193614 11452 leader_election.cc:302] T 1744600000b94366a308f5597aed4a6f P 16609e7c15b14fa7be968382cdc14db1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16609e7c15b14fa7be968382cdc14db1, e7aae91f56ad4fa1b1fcca3d3c9e01ed; no voters:
I20260809 15:09:28.193636 11109 raft_consensus.cc:2454] T 1744600000b94366a308f5597aed4a6f P ae69fba8b3fd4309855d72a1575005c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16609e7c15b14fa7be968382cdc14db1 in term 0.
I20260809 15:09:28.194047 11650 raft_consensus.cc:2781] T 1744600000b94366a308f5597aed4a6f P 16609e7c15b14fa7be968382cdc14db1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:09:28.194224 11650 raft_consensus.cc:482] T 1744600000b94366a308f5597aed4a6f P 16609e7c15b14fa7be968382cdc14db1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:09:28.194334 11650 raft_consensus.cc:3037] T 1744600000b94366a308f5597aed4a6f P 16609e7c15b14fa7be968382cdc14db1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:09:28.196662 11650 raft_consensus.cc:504] T 1744600000b94366a308f5597aed4a6f P 16609e7c15b14fa7be968382cdc14db1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7aae91f56ad4fa1b1fcca3d3c9e01ed" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 38443 } } peers { permanent_uuid: "ae69fba8b3fd4309855d72a1575005c1" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 45931 } } peers { permanent_uuid: "16609e7c15b14fa7be968382cdc14db1" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 46781 } }
I20260809 15:09:28.197214 11650 leader_election.cc:288] T 1744600000b94366a308f5597aed4a6f P 16609e7c15b14fa7be968382cdc14db1 [CANDIDATE]: Term 1 election: Requested vote from peers e7aae91f56ad4fa1b1fcca3d3c9e01ed (127.24.183.194:38443), ae69fba8b3fd4309855d72a1575005c1 (127.24.183.193:45931)
I20260809 15:09:28.197705 11338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1744600000b94366a308f5597aed4a6f" candidate_uuid: "16609e7c15b14fa7be968382cdc14db1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7aae91f56ad4fa1b1fcca3d3c9e01ed"
I20260809 15:09:28.197808 11109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1744600000b94366a308f5597aed4a6f" candidate_uuid: "16609e7c15b14fa7be968382cdc14db1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae69fba8b3fd4309855d72a1575005c1"
I20260809 15:09:28.197906 11338 raft_consensus.cc:3037] T 1744600000b94366a308f5597aed4a6f P e7aae91f56ad4fa1b1fcca3d3c9e01ed [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:09:28.198017 11109 raft_consensus.cc:3037] T 1744600000b94366a308f5597aed4a6f P ae69fba8b3fd4309855d72a1575005c1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:09:28.200220 11338 raft_consensus.cc:2454] T 1744600000b94366a308f5597aed4a6f P e7aae91f56ad4fa1b1fcca3d3c9e01ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16609e7c15b14fa7be968382cdc14db1 in term 1.
I20260809 15:09:28.200465 11109 raft_consensus.cc:2454] T 1744600000b94366a308f5597aed4a6f P ae69fba8b3fd4309855d72a1575005c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16609e7c15b14fa7be968382cdc14db1 in term 1.
I20260809 15:09:28.200578 11452 leader_election.cc:302] T 1744600000b94366a308f5597aed4a6f P 16609e7c15b14fa7be968382cdc14db1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16609e7c15b14fa7be968382cdc14db1, e7aae91f56ad4fa1b1fcca3d3c9e01ed; no voters:
I20260809 15:09:28.200877 11650 raft_consensus.cc:2781] T 1744600000b94366a308f5597aed4a6f P 16609e7c15b14fa7be968382cdc14db1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:09:28.201329 11650 raft_consensus.cc:686] T 1744600000b94366a308f5597aed4a6f P 16609e7c15b14fa7be968382cdc14db1 [term 1 LEADER]: Becoming Leader. State: Replica: 16609e7c15b14fa7be968382cdc14db1, State: Running, Role: LEADER
I20260809 15:09:28.201865 11650 consensus_queue.cc:237] T 1744600000b94366a308f5597aed4a6f P 16609e7c15b14fa7be968382cdc14db1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7aae91f56ad4fa1b1fcca3d3c9e01ed" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 38443 } } peers { permanent_uuid: "ae69fba8b3fd4309855d72a1575005c1" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 45931 } } peers { permanent_uuid: "16609e7c15b14fa7be968382cdc14db1" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 46781 } }
I20260809 15:09:28.206480 10858 catalog_manager.cc:5495] T 1744600000b94366a308f5597aed4a6f P 16609e7c15b14fa7be968382cdc14db1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 16609e7c15b14fa7be968382cdc14db1 (127.24.183.195). New cstate: current_term: 1 leader_uuid: "16609e7c15b14fa7be968382cdc14db1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7aae91f56ad4fa1b1fcca3d3c9e01ed" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 38443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae69fba8b3fd4309855d72a1575005c1" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 45931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16609e7c15b14fa7be968382cdc14db1" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260809 15:09:28.260505 11633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:09:28.254976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53270 (local address 127.24.183.195:46781)
0809 15:09:28.255194 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 15:09:28.255200 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:09:28.255219 (+ 19us) server_negotiation.cc:407] Connection header received
0809 15:09:28.255316 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:09:28.255328 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:09:28.255442 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:09:28.255576 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:09:28.256001 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:09:28.256758 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:09:28.258255 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:09:28.258687 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:09:28.258797 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:09:28.259311 (+ 514us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:09:28.259901 (+ 590us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:09:28.260058 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 15:09:28.260170 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 15:09:28.260255 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 15:09:28.268774 11109 raft_consensus.cc:1261] T 1744600000b94366a308f5597aed4a6f P ae69fba8b3fd4309855d72a1575005c1 [term 1 FOLLOWER]: Refusing update from remote peer 16609e7c15b14fa7be968382cdc14db1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:09:28.268775 11338 raft_consensus.cc:1261] T 1744600000b94366a308f5597aed4a6f P e7aae91f56ad4fa1b1fcca3d3c9e01ed [term 1 FOLLOWER]: Refusing update from remote peer 16609e7c15b14fa7be968382cdc14db1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:09:28.269686 11654 consensus_queue.cc:1035] T 1744600000b94366a308f5597aed4a6f P 16609e7c15b14fa7be968382cdc14db1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae69fba8b3fd4309855d72a1575005c1" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 45931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:09:28.270076 11650 consensus_queue.cc:1035] T 1744600000b94366a308f5597aed4a6f P 16609e7c15b14fa7be968382cdc14db1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7aae91f56ad4fa1b1fcca3d3c9e01ed" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 38443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:09:28.282539 11661 mvcc.cc:205] Tried to move back new op lower bound from 7316636337218523136 to 7316636336958812160. Current Snapshot: MvccSnapshot[applied={T|T < 7316636337218523136 or (T in {7316636337218523136})}]
I20260809 15:09:28.298107 10858 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:09:28.298354 10858 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:09:28.301584 10858 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c1256b8a7f38443097164f9a8a6b2484: Sending DeleteTablet for 3 replicas of tablet 1744600000b94366a308f5597aed4a6f
I20260809 15:09:28.302744 11306 tablet_service.cc:1514] Processing DeleteTablet for tablet 1744600000b94366a308f5597aed4a6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:09:28 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:39934
I20260809 15:09:28.302918 11517 tablet_service.cc:1514] Processing DeleteTablet for tablet 1744600000b94366a308f5597aed4a6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:09:28 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:53260
I20260809 15:09:28.302891 11082 tablet_service.cc:1514] Processing DeleteTablet for tablet 1744600000b94366a308f5597aed4a6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:09:28 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:42548
I20260809 15:09:28.304684 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 10969
I20260809 15:09:28.315608 11673 tablet_replica.cc:330] T 1744600000b94366a308f5597aed4a6f P e7aae91f56ad4fa1b1fcca3d3c9e01ed: stopping tablet replica
I20260809 15:09:28.316581 11673 raft_consensus.cc:2229] T 1744600000b94366a308f5597aed4a6f P e7aae91f56ad4fa1b1fcca3d3c9e01ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:09:28.317775 11673 raft_consensus.cc:2258] T 1744600000b94366a308f5597aed4a6f P e7aae91f56ad4fa1b1fcca3d3c9e01ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:09:28.318423 11674 tablet_replica.cc:330] T 1744600000b94366a308f5597aed4a6f P ae69fba8b3fd4309855d72a1575005c1: stopping tablet replica
I20260809 15:09:28.318830 11674 raft_consensus.cc:2229] T 1744600000b94366a308f5597aed4a6f P ae69fba8b3fd4309855d72a1575005c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:09:28.319367 11674 raft_consensus.cc:2258] T 1744600000b94366a308f5597aed4a6f P ae69fba8b3fd4309855d72a1575005c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:09:28.320212 11675 tablet_replica.cc:330] T 1744600000b94366a308f5597aed4a6f P 16609e7c15b14fa7be968382cdc14db1: stopping tablet replica
I20260809 15:09:28.321031 11673 ts_tablet_manager.cc:1905] T 1744600000b94366a308f5597aed4a6f P e7aae91f56ad4fa1b1fcca3d3c9e01ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:09:28.321506 11674 ts_tablet_manager.cc:1905] T 1744600000b94366a308f5597aed4a6f P ae69fba8b3fd4309855d72a1575005c1: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 15:09:28 dist-test-slave-nj21 krb5kdc[10075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.193@KRBTEST.COM
I20260809 15:09:28.328073 11673 ts_tablet_manager.cc:1918] T 1744600000b94366a308f5597aed4a6f P e7aae91f56ad4fa1b1fcca3d3c9e01ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:09:28.328132 11675 raft_consensus.cc:2229] T 1744600000b94366a308f5597aed4a6f P 16609e7c15b14fa7be968382cdc14db1 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:09:28.328279 11673 log.cc:1196] T 1744600000b94366a308f5597aed4a6f P e7aae91f56ad4fa1b1fcca3d3c9e01ed: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-1/wal/wals/1744600000b94366a308f5597aed4a6f
I20260809 15:09:28.328637 11674 ts_tablet_manager.cc:1918] T 1744600000b94366a308f5597aed4a6f P ae69fba8b3fd4309855d72a1575005c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:09:28.328811 11673 ts_tablet_manager.cc:1939] T 1744600000b94366a308f5597aed4a6f P e7aae91f56ad4fa1b1fcca3d3c9e01ed: Deleting consensus metadata
I20260809 15:09:28.328895 11674 log.cc:1196] T 1744600000b94366a308f5597aed4a6f P ae69fba8b3fd4309855d72a1575005c1: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-0/wal/wals/1744600000b94366a308f5597aed4a6f
I20260809 15:09:28.328926 11675 raft_consensus.cc:2258] T 1744600000b94366a308f5597aed4a6f P 16609e7c15b14fa7be968382cdc14db1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:09:28.329548 11674 ts_tablet_manager.cc:1939] T 1744600000b94366a308f5597aed4a6f P ae69fba8b3fd4309855d72a1575005c1: Deleting consensus metadata
I20260809 15:09:28.330813 10837 catalog_manager.cc:4841] TS e7aae91f56ad4fa1b1fcca3d3c9e01ed (127.24.183.194:38443): tablet 1744600000b94366a308f5597aed4a6f (table test-table [id=05160d2ec85c4f399d6ece1a7f87bcb7]) successfully deleted
I20260809 15:09:28.331960 11675 ts_tablet_manager.cc:1905] T 1744600000b94366a308f5597aed4a6f P 16609e7c15b14fa7be968382cdc14db1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:09:28.335897 11632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:09:28.306250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42560 (local address 127.24.183.193:45931)
0809 15:09:28.306512 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0809 15:09:28.306521 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:09:28.314602 (+ 8081us) server_negotiation.cc:407] Connection header received
0809 15:09:28.315339 (+ 737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:09:28.315345 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:09:28.315434 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:09:28.315551 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:09:28.317575 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:09:28.318937 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:09:28.321093 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:09:28.321793 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:09:28.321918 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:09:28.327901 (+ 5983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:09:28.327918 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:09:28.327936 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:09:28.327983 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:09:28.332443 (+ 4460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:09:28.333328 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:09:28.333337 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:09:28.333341 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:09:28.333515 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:09:28.333965 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:09:28.333971 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:09:28.333975 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:09:28.334462 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:09:28.334800 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0809 15:09:28.335130 (+ 330us) server_negotiation.cc:299] Negotiation successful
0809 15:09:28.335403 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 15:09:28.336702 11172 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:09:28.340190 11675 ts_tablet_manager.cc:1918] T 1744600000b94366a308f5597aed4a6f P 16609e7c15b14fa7be968382cdc14db1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:09:28.340376 11675 log.cc:1196] T 1744600000b94366a308f5597aed4a6f P 16609e7c15b14fa7be968382cdc14db1: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288021313853-25311-0/minicluster-data/ts-2/wal/wals/1744600000b94366a308f5597aed4a6f
I20260809 15:09:28.341080 11675 ts_tablet_manager.cc:1939] T 1744600000b94366a308f5597aed4a6f P 16609e7c15b14fa7be968382cdc14db1: Deleting consensus metadata
I20260809 15:09:28.342896 10837 catalog_manager.cc:4841] TS 16609e7c15b14fa7be968382cdc14db1 (127.24.183.195:46781): tablet 1744600000b94366a308f5597aed4a6f (table test-table [id=05160d2ec85c4f399d6ece1a7f87bcb7]) successfully deleted
I20260809 15:09:28.428082 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 10969
I20260809 15:09:28.428871 10837 catalog_manager.cc:4841] TS ae69fba8b3fd4309855d72a1575005c1 (127.24.183.193:45931): tablet 1744600000b94366a308f5597aed4a6f (table test-table [id=05160d2ec85c4f399d6ece1a7f87bcb7]) successfully deleted
I20260809 15:09:28.450965 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 11187
Aug 09 15:09:28 dist-test-slave-nj21 krb5kdc[10075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.194@KRBTEST.COM
I20260809 15:09:28.467857 11629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:09:28.451836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39940 (local address 127.24.183.194:38443)
0809 15:09:28.452077 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 15:09:28.452083 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:09:28.452107 (+ 24us) server_negotiation.cc:407] Connection header received
0809 15:09:28.452214 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:09:28.452220 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:09:28.452306 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:09:28.452415 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:09:28.453631 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:09:28.454745 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:09:28.456052 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:09:28.456684 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:09:28.456816 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:09:28.461080 (+ 4264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:09:28.461099 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:09:28.461121 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:09:28.461178 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:09:28.464950 (+ 3772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:09:28.465570 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:09:28.465580 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:09:28.465584 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:09:28.465750 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:09:28.466061 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:09:28.466066 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:09:28.466070 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:09:28.466584 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:09:28.466903 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0809 15:09:28.467202 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 15:09:28.467434 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260809 15:09:28.468595 11399 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:09:28.557046 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 11187
I20260809 15:09:28.589669 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 11415
Aug 09 15:09:28 dist-test-slave-nj21 krb5kdc[10075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.195@KRBTEST.COM
I20260809 15:09:28.607283 11633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:09:28.590523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53278 (local address 127.24.183.195:46781)
0809 15:09:28.590732 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 15:09:28.590739 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:09:28.590763 (+ 24us) server_negotiation.cc:407] Connection header received
0809 15:09:28.590840 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:09:28.590845 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:09:28.590932 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:09:28.591042 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:09:28.592378 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:09:28.593620 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:09:28.594923 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:09:28.595601 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:09:28.595897 (+ 296us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:09:28.600009 (+ 4112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:09:28.600032 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:09:28.600055 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:09:28.600099 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:09:28.604411 (+ 4312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:09:28.605157 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:09:28.605167 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:09:28.605171 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:09:28.605367 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:09:28.605693 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:09:28.605698 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:09:28.605702 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:09:28.606176 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:09:28.606499 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0809 15:09:28.606658 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 15:09:28.606884 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 15:09:28.608389 11605 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:09:28.734093 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 11415
I20260809 15:09:28.759883 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 10810
I20260809 15:09:28.761106 10900 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:09:28.875541 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 10810
I20260809 15:09:28.902367 25311 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 15:09:29.246837 25311 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 15:09:29.247126 25311 mini_ranger.cc:66] Stopping Ranger...
I20260809 15:09:29.591323 25311 mini_ranger.cc:68] Stopped Ranger
2026-08-09 15:09:29.591 UTC [10123] LOG: received smart shutdown request
2026-08-09 15:09:29.595 UTC [10123] LOG: background worker "logical replication launcher" (PID 10136) exited with exit code 1
2026-08-09 15:09:29.595 UTC [10128] LOG: shutting down
2026-08-09 15:09:29.741 UTC [10123] LOG: database system is shut down
2026-08-09T15:09:29Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (49176 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:09:29 dist-test-slave-nj21 krb5kdc[11695](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:09:29 dist-test-slave-nj21 krb5kdc[11695](info): set up 2 sockets
Aug 09 15:09:29 dist-test-slave-nj21 krb5kdc[11695](info): commencing operation
krb5kdc: starting...
W20260809 15:09:31.076488 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.005s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:09:31 dist-test-slave-nj21 krb5kdc[11695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:09:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:09:31Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.24.183.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.24.183.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.24.183.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/rangeradmin_127.24.183.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.24.183.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/rangeradmin_127.24.183.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.24.183.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.24.183.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.24.183.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/rangerlookup_127.24.183.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.24.183.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/rangerlookup_127.24.183.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.183.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.24.183.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.183.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/HTTP_127.24.183.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.183.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/HTTP_127.24.183.212@KRBTEST.COM.keytab.
I20260809 15:09:31.438195 25311 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskXnZFlc/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/postgres -l logfile start
2026-08-09 15:09:33.740 UTC [11738] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 15:09:33.741 UTC [11738] LOG: listening on IPv4 address "127.24.183.212", port 55677
2026-08-09 15:09:33.744 UTC [11738] LOG: listening on Unix socket "/tmp/.s.PGSQL.55677"
2026-08-09 15:09:33.764 UTC [11742] LOG: database system was shut down at 2026-08-09 15:09:32 UTC
2026-08-09 15:09:33.768 UTC [11738] LOG: database system is ready to accept connections
I20260809 15:09:34.853608 25311 mini_postgres.cc:96] Postgres bound to 55677
2026-08-09 15:09:34.880 UTC [11751] FATAL: database "slave" does not exist
127.24.183.212:55677 - accepting connections
I20260809 15:09:34.880759 25311 mini_ranger.cc:161] Starting Ranger...
I20260809 15:09:34.907843 25311 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 15:09:35.362673 25311 mini_ranger.cc:87] Created ranger Postgres database
I20260809 15:09:35.362843 25311 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ranger-admin
2026-08-09 15:09:35,644 [I] DB FLAVOR :POSTGRES
2026-08-09 15:09:35,644 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 15:09:35,644 [I] Checking connection..
2026-08-09 15:09:35,645 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:55677/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 15:09:35,903 [I] Checking connection passed.
2026-08-09 15:09:35,903 [I] --------- Verifying version history table ---------
2026-08-09 15:09:35,903 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:55677/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 15:09:36,167 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 15:09:36,167 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:55677/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 15:09:36,425 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 15:09:36,425 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 15:09:36,425 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:55677/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 15:09:36.666 UTC [11839] WARNING: there is no transaction in progress
2026-08-09 15:09:36,675 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 15:09:36,675 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:55677/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 15:09:36,943 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 15:09:36,943 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 15:09:36,943 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:55677/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 15:09:37,199 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:55677/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 15:09:37,451 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:55677/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-nj21.c.gcp-upstream.internal','N') ;"
2026-08-09 15:09:37,707 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:55677/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 15:09:37,963 [I] Table x_portal_user does not exist in database ranger
2026-08-09 15:09:37,963 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:55677/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 15:09:38,206 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 15:09:38,206 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:55677/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 15:09:38,466 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:55677/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 15:09:38,711 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 15:09:38,711 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:55677/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 15:09:39.599 UTC [12000] WARNING: there is no transaction in progress
2026-08-09 15:09:39.614 UTC [12000] WARNING: there is no transaction in progress
2026-08-09 15:09:39.630 UTC [12000] WARNING: there is no transaction in progress
2026-08-09 15:09:39.644 UTC [12000] WARNING: there is no transaction in progress
2026-08-09 15:09:39.659 UTC [12000] WARNING: there is no transaction in progress
2026-08-09 15:09:39.679 UTC [12000] WARNING: there is no transaction in progress
2026-08-09 15:09:39.731 UTC [12000] WARNING: there is no transaction in progress
2026-08-09 15:09:39.737 UTC [12000] WARNING: there is no transaction in progress
2026-08-09 15:09:39.750 UTC [12000] WARNING: there is no transaction in progress
2026-08-09 15:09:39.759 UTC [12000] WARNING: there is no transaction in progress
2026-08-09 15:09:39.768 UTC [12000] WARNING: there is no transaction in progress
2026-08-09 15:09:39.781 UTC [12000] WARNING: there is no transaction in progress
2026-08-09 15:09:39.790 UTC [12000] WARNING: there is no transaction in progress
2026-08-09 15:09:39.796 UTC [12000] WARNING: there is no transaction in progress
2026-08-09 15:09:39.802 UTC [12000] WARNING: there is no transaction in progress
2026-08-09 15:09:39.808 UTC [12000] WARNING: there is no transaction in progress
2026-08-09 15:09:40,396 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 15:09:40,396 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:55677/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 15:09:40,654 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 15:09:40,655 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:55677/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 15:09:40,914 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 15:09:40,914 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:55677/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 15:09:41,171 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 15:09:41,171 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:55677/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 15:09:41,414 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:55677/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 15:09:41,671 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:55677/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-nj21.c.gcp-upstream.internal';"
2026-08-09 15:09:41,903 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 15:09:41,903 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:55677/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 15:09:42,159 [I] DB_PATCHES have already been applied
I20260809 15:09:42.165378 25311 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ranger-admin:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskXnZFlc/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 15:09:42.165464 25311 mini_ranger.cc:193] Using host: 127.24.183.212
I20260809 15:09:42.185148 25311 mini_ranger.cc:227] Ranger admin URL: http://127.24.183.212:56835
Aug 09, 2026 3:09:42 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 3:09:42 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 3:09:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 3:09:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 3:09:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 3:09:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 3:09:43 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 15:09:43.464112 25311 mini_ranger.cc:160] Time spent starting Ranger: real 8.583s user 0.004s sys 0.050s
Aug 09, 2026 3:10:02 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 3:10:02 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 09, 2026 3:10:02 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 3:10:02 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 3:10:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 3:10:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 3:10:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 3:10:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 3:10:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 3:10:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 3:10:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 3:10:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 3:10:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 3:10:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 3:10:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 3:10:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 3:10:02 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 3:10:03 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 15:10:03.853513 25311 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.24.183.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.24.183.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.24.183.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/rangerkms_127.24.183.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.24.183.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/rangerkms_127.24.183.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.183.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.24.183.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.183.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/HTTP_127.24.183.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.183.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/HTTP_127.24.183.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 15:10:04 dist-test-slave-nj21 krb5kdc[11695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288204, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 15:10:04.055135 25311 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 15:10:04.080039 25311 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 15:10:04.927522 25311 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 15:10:04.927649 25311 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ranger-kms
I20260809 15:10:04.927709 25311 mini_ranger_kms.cc:227] Using postgres at 127.24.183.212:55677
2026-08-09 15:10:05,291 [I] DB FLAVOR :POSTGRES
2026-08-09 15:10:05,291 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 15:10:05,292 [I] Checking connection
2026-08-09 15:10:05,292 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:55677/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 15:10:05,533 [I] connection success
2026-08-09 15:10:05,533 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 15:10:05,533 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 15:10:05,533 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:55677/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 15:10:05,782 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 15:10:05,782 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 15:10:05,783 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 15:10:05,783 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.183.212:55677/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 15:10:06,046 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 15:10:06.527108 25311 mini_ranger_kms.cc:310] Created kms service
I20260809 15:10:06.756477 25311 mini_ranger_kms.cc:326] Created kudu user
I20260809 15:10:06.808717 25311 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 15:10:06 dist-test-slave-nj21 krb5kdc[11695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288204, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.24.183.212@KRBTEST.COM
I20260809 15:10:07.209395 25311 mini_ranger_kms.cc:384] Added ranger policy
I20260809 15:10:07.209743 25311 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ranger-kms:/tmp/dist-test-taskXnZFlc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskXnZFlc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 15:10:07.209810 25311 mini_ranger_kms.cc:240] Using host: 127.24.183.212
I20260809 15:10:07.231568 25311 mini_ranger_kms.cc:276] Ranger KMS PID: 12285
I20260809 15:10:07.231729 25311 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.24.183.212:40483
Aug 09, 2026 3:10:08 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 3:10:08 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 3:10:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 3:10:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 3:10:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 3:10:08 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 15:10:09 dist-test-slave-nj21 krb5kdc[11695](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786288209, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.183.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:10:09.793166 25311 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.738s user 0.085s sys 0.316s
I20260809 15:10:09.793430 25311 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 15:10:09.793594 25311 mini_ranger_kms.cc:401] 127.24.183.212:40483/kms/v1/keys
Aug 09 15:10:10 dist-test-slave-nj21 krb5kdc[11695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288209, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.183.212@KRBTEST.COM for HTTP/127.24.183.212@KRBTEST.COM
Aug 09 15:10:10 dist-test-slave-nj21 krb5kdc[11695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288209, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.183.212@KRBTEST.COM for HTTP/127.24.183.212@KRBTEST.COM
Aug 09 15:10:12 dist-test-slave-nj21 krb5kdc[11695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288212, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.24.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
I20260809 15:10:12.732004 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:34693
--webserver_interface=127.24.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:43755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:34693
--ranger_config_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.183.212:40483/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:10:13.073734 12362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:10:13.074028 12362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:10:13.074165 12362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:10:13.081334 12362 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:10:13.081450 12362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:10:13.081532 12362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:10:13.081605 12362 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:10:13.081674 12362 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:10:13.091562 12362 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:43755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/master-0/wal
--ranger_kms_url=127.24.183.212:40483/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:34693
--ranger_config_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:34693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:10:13.092847 12362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:10:13.094177 12362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:10:13.105892 12370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:13.105854 12371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:13.105854 12375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:10:13.106961 12362 server_base.cc:1034] running on GCE node
I20260809 15:10:13.107785 12362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:10:13.109155 12362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:10:13.110357 12362 hybrid_clock.cc:648] HybridClock initialized: now 1786288213110315 us; error 29 us; skew 500 ppm
I20260809 15:10:13.110733 12362 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:10:13 dist-test-slave-nj21 krb5kdc[11695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288213, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:10:13.118134 12362 init.cc:377] Logged in from keytab as kudu/127.24.183.254@KRBTEST.COM (short username kudu)
I20260809 15:10:13.119593 12362 webserver.cc:460] Webserver started at http://127.24.183.254:39229/ using document root <none> and password file <none>
I20260809 15:10:13.120266 12362 fs_manager.cc:359] Metadata directory not provided
I20260809 15:10:13.120371 12362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:10:13.120643 12362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 15:10:13 dist-test-slave-nj21 krb5kdc[11695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288213, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for HTTP/127.24.183.212@KRBTEST.COM
I20260809 15:10:13.180778 12362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "4b471d8ade814391a684ef3b622018e5"
format_stamp: "Formatted at 2026-08-09 15:10:13 on dist-test-slave-nj21"
server_key: "b6ab40ef346807c44f46170bdb5caeb3"
server_key_iv: "96c28a4dc76f26c7bcc2ae62372a350b"
server_key_version: "kuduclusterkey@0"
I20260809 15:10:13.181672 12362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "4b471d8ade814391a684ef3b622018e5"
format_stamp: "Formatted at 2026-08-09 15:10:13 on dist-test-slave-nj21"
server_key: "b6ab40ef346807c44f46170bdb5caeb3"
server_key_iv: "96c28a4dc76f26c7bcc2ae62372a350b"
server_key_version: "kuduclusterkey@0"
I20260809 15:10:13.187351 12362 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260809 15:10:13.207448 12384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:13.208925 12362 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:10:13.209108 12362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "4b471d8ade814391a684ef3b622018e5"
format_stamp: "Formatted at 2026-08-09 15:10:13 on dist-test-slave-nj21"
server_key: "b6ab40ef346807c44f46170bdb5caeb3"
server_key_iv: "96c28a4dc76f26c7bcc2ae62372a350b"
server_key_version: "kuduclusterkey@0"
I20260809 15:10:13.209334 12362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:10:13.238365 12362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:10:13.242029 12362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:10:13.242311 12362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:10:13.259603 12362 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:34693
I20260809 15:10:13.259624 12465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:34693 every 8 connection(s)
I20260809 15:10:13.260924 12362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:10:13.264559 12466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:10:13.270793 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 12362
I20260809 15:10:13.271014 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:10:13.271898 25311 external_mini_cluster.cc:1428] Setting key 9c816ac51e422dee656c3d21f1768499
I20260809 15:10:13.275914 12466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5: Bootstrap starting.
I20260809 15:10:13.279819 12466 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5: Neither blocks nor log segments found. Creating new log.
I20260809 15:10:13.281169 12466 log.cc:824] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5: Log is configured to *not* fsync() on all Append() calls
I20260809 15:10:13.284288 12466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5: No bootstrap required, opened a new log
Aug 09 15:10:13 dist-test-slave-nj21 krb5kdc[11695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288212, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:10:13.293329 12466 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b471d8ade814391a684ef3b622018e5" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 34693 } }
I20260809 15:10:13.293820 12466 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:10:13.293947 12466 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b471d8ade814391a684ef3b622018e5, State: Initialized, Role: FOLLOWER
I20260809 15:10:13.294651 12466 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b471d8ade814391a684ef3b622018e5" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 34693 } }
I20260809 15:10:13.294993 12466 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:10:13.295151 12466 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:10:13.295325 12466 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:10:13.296546 12470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:13.274139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59912 (local address 127.24.183.254:34693)
0809 15:10:13.275022 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:13.275039 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:13.275363 (+ 324us) server_negotiation.cc:407] Connection header received
0809 15:10:13.277809 (+ 2446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:13.277840 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:13.278320 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:13.278722 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:13.279925 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:13.281894 (+ 1969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:13.283323 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:13.284336 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:13.284526 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:13.288155 (+ 3629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:13.288165 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:13.288183 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:13.288243 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:13.292079 (+ 3836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:13.292736 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:13.292744 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:13.292752 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:13.292938 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:13.293260 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:13.293265 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:13.293268 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:13.294014 (+ 746us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:13.294310 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:13.294702 (+ 392us) server_negotiation.cc:299] Negotiation successful
0809 15:10:13.295126 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":378,"threads_started":1}
I20260809 15:10:13.298221 12466 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b471d8ade814391a684ef3b622018e5" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 34693 } }
I20260809 15:10:13.298722 12466 leader_election.cc:302] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b471d8ade814391a684ef3b622018e5; no voters:
I20260809 15:10:13.299337 12466 leader_election.cc:288] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:10:13.299437 12473 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:10:13.300949 12473 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5 [term 1 LEADER]: Becoming Leader. State: Replica: 4b471d8ade814391a684ef3b622018e5, State: Running, Role: LEADER
I20260809 15:10:13.301409 12466 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:10:13.301430 12473 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b471d8ade814391a684ef3b622018e5" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 34693 } }
I20260809 15:10:13.304430 12474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b471d8ade814391a684ef3b622018e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b471d8ade814391a684ef3b622018e5" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 34693 } } }
I20260809 15:10:13.304417 12475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b471d8ade814391a684ef3b622018e5. Latest consensus state: current_term: 1 leader_uuid: "4b471d8ade814391a684ef3b622018e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b471d8ade814391a684ef3b622018e5" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 34693 } } }
I20260809 15:10:13.304816 12474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5 [sys.catalog]: This master's current role is: LEADER
I20260809 15:10:13.304862 12475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5 [sys.catalog]: This master's current role is: LEADER
I20260809 15:10:13.305721 12490 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:10:13.311699 12490 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:10:13.316572 12466 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 15:10:13.333125 12490 catalog_manager.cc:1347] Generated new cluster ID: 09b1a38cf52f43f4a495c828e5f95bbd
I20260809 15:10:13.333267 12490 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:10:13.344977 12490 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:10:13.346915 12490 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:10:13.355641 12490 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5: Generated new TSK 0
I20260809 15:10:13.356670 12490 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab.
Entry for principal kudu/127.24.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab.
I20260809 15:10:14.103250 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.183.212:40483/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.193:0
--local_ip_for_outbound_sockets=127.24.183.193
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:34693
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:43755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 15:10:14 dist-test-slave-nj21 krb5kdc[11695](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786288214, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 15:10:14.556448 12535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:10:14.556864 12535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:10:14.557089 12535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:10:14.568578 12535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:10:14.568724 12535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:10:14.568905 12535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.193
I20260809 15:10:14.581439 12535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:43755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.24.183.212:40483/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:34693
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.193
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:10:14.583284 12535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:10:14.584816 12535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:10:14.603633 12550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:14.604417 12551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:14.612192 12553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:10:14.613020 12535 server_base.cc:1034] running on GCE node
I20260809 15:10:14.613931 12535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:10:14.615257 12535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:10:14.616539 12535 hybrid_clock.cc:648] HybridClock initialized: now 1786288214616469 us; error 51 us; skew 500 ppm
I20260809 15:10:14.616907 12535 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:10:14 dist-test-slave-nj21 krb5kdc[11695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288214, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:10:14.620857 12535 init.cc:377] Logged in from keytab as kudu/127.24.183.193@KRBTEST.COM (short username kudu)
I20260809 15:10:14.622709 12535 webserver.cc:460] Webserver started at http://127.24.183.193:45413/ using document root <none> and password file <none>
I20260809 15:10:14.623660 12535 fs_manager.cc:359] Metadata directory not provided
I20260809 15:10:14.623834 12535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:10:14.624212 12535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 15:10:14 dist-test-slave-nj21 krb5kdc[11695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288214, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for HTTP/127.24.183.212@KRBTEST.COM
I20260809 15:10:14.641767 12535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-0/data/instance:
uuid: "69cbebcd7b864b18a5e8865f5e479bad"
format_stamp: "Formatted at 2026-08-09 15:10:14 on dist-test-slave-nj21"
server_key: "6d04ceae9e8c6fdafdc3a939cb8ea8dc"
server_key_iv: "a1edc2ea949a4a15fdde5b7f6fd6f14d"
server_key_version: "kuduclusterkey@0"
I20260809 15:10:14.642637 12535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance:
uuid: "69cbebcd7b864b18a5e8865f5e479bad"
format_stamp: "Formatted at 2026-08-09 15:10:14 on dist-test-slave-nj21"
server_key: "6d04ceae9e8c6fdafdc3a939cb8ea8dc"
server_key_iv: "a1edc2ea949a4a15fdde5b7f6fd6f14d"
server_key_version: "kuduclusterkey@0"
I20260809 15:10:14.680001 12535 fs_manager.cc:691] Time spent creating directory manager: real 0.037s user 0.019s sys 0.017s
I20260809 15:10:14.712347 12565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:14.714555 12535 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.004s sys 0.004s
I20260809 15:10:14.714789 12535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-0/wal
uuid: "69cbebcd7b864b18a5e8865f5e479bad"
format_stamp: "Formatted at 2026-08-09 15:10:14 on dist-test-slave-nj21"
server_key: "6d04ceae9e8c6fdafdc3a939cb8ea8dc"
server_key_iv: "a1edc2ea949a4a15fdde5b7f6fd6f14d"
server_key_version: "kuduclusterkey@0"
I20260809 15:10:14.715114 12535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:10:14.740644 12535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:10:14.746106 12535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:10:14.746496 12535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:10:14.748111 12535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:10:14.750051 12535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:10:14.750159 12535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:14.750319 12535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:10:14.750399 12535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:14.800901 12748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.193:33831 every 8 connection(s)
I20260809 15:10:14.801746 12535 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.193:33831
I20260809 15:10:14.804651 12535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
I20260809 15:10:14.812950 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 12535
I20260809 15:10:14.813192 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance
I20260809 15:10:14.814229 25311 external_mini_cluster.cc:1428] Setting key 472ee484b4a645f0d7e98313e1a482f6
Aug 09 15:10:14 dist-test-slave-nj21 krb5kdc[11695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288214, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for HTTP/127.24.183.212@KRBTEST.COM
Aug 09 15:10:14 dist-test-slave-nj21 krb5kdc[11695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288214, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.183.194@KRBTEST.COM; defaulting to no policy
I20260809 15:10:14.894662 12758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:14.836007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:60359 (local address 127.24.183.254:34693)
0809 15:10:14.864267 (+ 28260us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:14.864275 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:14.867433 (+ 3158us) server_negotiation.cc:407] Connection header received
0809 15:10:14.867498 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:14.867502 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:14.867593 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:14.867701 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:14.869710 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:14.870465 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:14.872784 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:14.873363 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:14.873479 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:14.877560 (+ 4081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:14.877566 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:14.877570 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:14.877627 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:14.885000 (+ 7373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:14.890243 (+ 5243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:14.890249 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:14.890252 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:14.890430 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:14.892129 (+ 1699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:14.892135 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:14.892138 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:14.892532 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:14.892680 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:14.893749 (+ 1069us) server_negotiation.cc:299] Negotiation successful
0809 15:10:14.894010 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28078,"thread_start_us":27935,"threads_started":1}
I20260809 15:10:14.895548 12756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:14.835206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:34693 (local address 127.24.183.193:60359)
0809 15:10:14.864961 (+ 29755us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:14.865022 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:14.867131 (+ 2109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:14.868024 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:14.868037 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:14.868709 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:10:14.869534 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:14.869558 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:14.870628 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:14.870633 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:14.872633 (+ 2000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:14.872650 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:14.873614 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:14.873623 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:14.873884 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:14.874624 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:10:14.874652 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:10:14.877288 (+ 2636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:10:14.885638 (+ 8350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:14.885649 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:14.885673 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:14.886289 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:14.891641 (+ 5352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:14.891646 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:14.891650 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:14.891967 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:14.892771 (+ 804us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:10:14.892780 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:10:14.893247 (+ 467us) client_negotiation.cc:769] Sending connection context
0809 15:10:14.893581 (+ 334us) client_negotiation.cc:240] Negotiation successful
0809 15:10:14.894167 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29403,"thread_start_us":29120,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.194@KRBTEST.COM" created.
I20260809 15:10:14.897881 12754 heartbeater.cc:347] Connected to a master server at 127.24.183.254:34693
I20260809 15:10:14.898221 12754 heartbeater.cc:464] Registering TS with master...
I20260809 15:10:14.899176 12754 heartbeater.cc:511] Master 127.24.183.254:34693 requested a full tablet report, sending...
I20260809 15:10:14.902165 12409 ts_manager.cc:194] Registered new tserver with Master: 69cbebcd7b864b18a5e8865f5e479bad (127.24.183.193:33831)
I20260809 15:10:14.904388 12409 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.193@KRBTEST.COM'} at 127.24.183.193:60359
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab.
Entry for principal kudu/127.24.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab.
W20260809 15:10:14.936417 25311 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.24.183.194: real 0.119s user 0.001s sys 0.023s
I20260809 15:10:14.937409 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.183.212:40483/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.194:0
--local_ip_for_outbound_sockets=127.24.183.194
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:34693
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:43755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 15:10:15 dist-test-slave-nj21 krb5kdc[11695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288214, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for HTTP/127.24.183.212@KRBTEST.COM
W20260809 15:10:15.446669 12762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:10:15.447180 12762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:10:15.447391 12762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:10:15.458590 12762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:10:15.458829 12762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:10:15.459072 12762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.194
I20260809 15:10:15.474859 12762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:43755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.24.183.212:40483/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:34693
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.194
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:10:15.476967 12762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:10:15.478673 12762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:10:15.515996 12775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:15.516475 12773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:15.518424 12779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:10:15.518893 12762 server_base.cc:1034] running on GCE node
I20260809 15:10:15.520022 12762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:10:15.528767 12762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:10:15.530077 12762 hybrid_clock.cc:648] HybridClock initialized: now 1786288215529966 us; error 101 us; skew 500 ppm
I20260809 15:10:15.530555 12762 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:10:15 dist-test-slave-nj21 krb5kdc[11695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288215, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:10:15.534770 12762 init.cc:377] Logged in from keytab as kudu/127.24.183.194@KRBTEST.COM (short username kudu)
I20260809 15:10:15.536710 12762 webserver.cc:460] Webserver started at http://127.24.183.194:37147/ using document root <none> and password file <none>
I20260809 15:10:15.537665 12762 fs_manager.cc:359] Metadata directory not provided
I20260809 15:10:15.537865 12762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:10:15.538213 12762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 15:10:15 dist-test-slave-nj21 krb5kdc[11695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288215, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for HTTP/127.24.183.212@KRBTEST.COM
I20260809 15:10:15.564379 12762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-1/data/instance:
uuid: "7c95d6e5bf4e4cf2b4e083d99c636c8c"
format_stamp: "Formatted at 2026-08-09 15:10:15 on dist-test-slave-nj21"
server_key: "3cac3217d5bb2f5e4a76c3532b7cbe7e"
server_key_iv: "c1ad2de230528344a3176e969e256234"
server_key_version: "kuduclusterkey@0"
I20260809 15:10:15.566061 12762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance:
uuid: "7c95d6e5bf4e4cf2b4e083d99c636c8c"
format_stamp: "Formatted at 2026-08-09 15:10:15 on dist-test-slave-nj21"
server_key: "3cac3217d5bb2f5e4a76c3532b7cbe7e"
server_key_iv: "c1ad2de230528344a3176e969e256234"
server_key_version: "kuduclusterkey@0"
I20260809 15:10:15.574828 12762 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.001s
I20260809 15:10:15.615274 12800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:15.617298 12762 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.007s sys 0.000s
I20260809 15:10:15.617522 12762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-1/wal
uuid: "7c95d6e5bf4e4cf2b4e083d99c636c8c"
format_stamp: "Formatted at 2026-08-09 15:10:15 on dist-test-slave-nj21"
server_key: "3cac3217d5bb2f5e4a76c3532b7cbe7e"
server_key_iv: "c1ad2de230528344a3176e969e256234"
server_key_version: "kuduclusterkey@0"
I20260809 15:10:15.617877 12762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:10:15.655628 12762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:10:15.661024 12762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:10:15.661376 12762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:10:15.662577 12762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:10:15.664402 12762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:10:15.664498 12762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:15.664669 12762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:10:15.664748 12762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:15.731053 12762 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.194:43401
I20260809 15:10:15.731097 12988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.194:43401 every 8 connection(s)
I20260809 15:10:15.732853 12762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
I20260809 15:10:15.740813 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 12762
I20260809 15:10:15.741108 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance
I20260809 15:10:15.741696 25311 external_mini_cluster.cc:1428] Setting key 1686183dff910574605ce97901569454
Aug 09 15:10:15 dist-test-slave-nj21 krb5kdc[11695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288215, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:10:15.771559 12992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:15.735969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:41845 (local address 127.24.183.254:34693)
0809 15:10:15.748134 (+ 12165us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:15.748139 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:15.748160 (+ 21us) server_negotiation.cc:407] Connection header received
0809 15:10:15.748251 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:15.748256 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:15.748333 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:15.748449 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:15.750415 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:15.751449 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:15.759160 (+ 7711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:15.759787 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:15.759904 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:15.764407 (+ 4503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:15.764413 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:15.764418 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:15.764460 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:15.767752 (+ 3292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:15.768692 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:15.768697 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:15.768700 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:15.768850 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:15.769402 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:15.769408 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:15.769411 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:15.769782 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:15.769901 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:15.770938 (+ 1037us) server_negotiation.cc:299] Negotiation successful
0809 15:10:15.771190 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12026,"thread_start_us":11860,"threads_started":1}
I20260809 15:10:15.773510 12993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:15.736826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:34693 (local address 127.24.183.194:41845)
0809 15:10:15.739171 (+ 2345us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:15.739229 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:15.741611 (+ 2382us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:15.748738 (+ 7127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:15.748751 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:15.749409 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:10:15.750219 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:15.750257 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:15.757006 (+ 6749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:15.757012 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:15.759003 (+ 1991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:15.759020 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:15.760261 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:15.760272 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:15.760582 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:15.761328 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:10:15.761363 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:10:15.764136 (+ 2773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:10:15.767959 (+ 3823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:15.767967 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:15.767984 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:15.768561 (+ 577us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:15.769020 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:15.769023 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:15.769027 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:15.769277 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:15.770007 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:10:15.770015 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:10:15.770488 (+ 473us) client_negotiation.cc:769] Sending connection context
0809 15:10:15.771891 (+ 1403us) client_negotiation.cc:240] Negotiation successful
0809 15:10:15.772329 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2030,"thread_start_us":1810,"threads_started":1}
I20260809 15:10:15.774966 12989 heartbeater.cc:347] Connected to a master server at 127.24.183.254:34693
I20260809 15:10:15.775305 12989 heartbeater.cc:464] Registering TS with master...
I20260809 15:10:15.776242 12989 heartbeater.cc:511] Master 127.24.183.254:34693 requested a full tablet report, sending...
I20260809 15:10:15.777904 12409 ts_manager.cc:194] Registered new tserver with Master: 7c95d6e5bf4e4cf2b4e083d99c636c8c (127.24.183.194:43401)
I20260809 15:10:15.779052 12409 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.194@KRBTEST.COM'} at 127.24.183.194:41845
WARNING: no policy specified for kudu/127.24.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab.
Entry for principal kudu/127.24.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab.
I20260809 15:10:15.818603 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.183.212:40483/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.195:0
--local_ip_for_outbound_sockets=127.24.183.195
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:34693
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:43755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:10:15.908344 12754 heartbeater.cc:503] Master 127.24.183.254:34693 was elected leader, sending a full tablet report...
W20260809 15:10:16.165381 12998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:10:16.165696 12998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:10:16.165843 12998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:10:16.173121 12998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:10:16.173244 12998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:10:16.173403 12998 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.195
I20260809 15:10:16.182960 12998 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:43755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.24.183.212:40483/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:34693
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.195
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:10:16.184337 12998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:10:16.185691 12998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:10:16.200085 13005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:16.200177 13006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:16.201197 13009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:10:16.202136 12998 server_base.cc:1034] running on GCE node
I20260809 15:10:16.202829 12998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:10:16.203850 12998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:10:16.205065 12998 hybrid_clock.cc:648] HybridClock initialized: now 1786288216205003 us; error 55 us; skew 500 ppm
I20260809 15:10:16.205384 12998 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:10:16 dist-test-slave-nj21 krb5kdc[11695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288216, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:10:16.208477 12998 init.cc:377] Logged in from keytab as kudu/127.24.183.195@KRBTEST.COM (short username kudu)
I20260809 15:10:16.210095 12998 webserver.cc:460] Webserver started at http://127.24.183.195:38955/ using document root <none> and password file <none>
I20260809 15:10:16.210763 12998 fs_manager.cc:359] Metadata directory not provided
I20260809 15:10:16.210891 12998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:10:16.211189 12998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 15:10:16 dist-test-slave-nj21 krb5kdc[11695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288216, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for HTTP/127.24.183.212@KRBTEST.COM
I20260809 15:10:16.231619 12998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-2/data/instance:
uuid: "aa0cc61b53e84b7994ded5e2e7c0adf4"
format_stamp: "Formatted at 2026-08-09 15:10:16 on dist-test-slave-nj21"
server_key: "504966b60e555af6b46b64b47dfe5cff"
server_key_iv: "860c1aa96db184701fe203a25f537e35"
server_key_version: "kuduclusterkey@0"
I20260809 15:10:16.232730 12998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance:
uuid: "aa0cc61b53e84b7994ded5e2e7c0adf4"
format_stamp: "Formatted at 2026-08-09 15:10:16 on dist-test-slave-nj21"
server_key: "504966b60e555af6b46b64b47dfe5cff"
server_key_iv: "860c1aa96db184701fe203a25f537e35"
server_key_version: "kuduclusterkey@0"
I20260809 15:10:16.238806 12998 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260809 15:10:16.255811 13017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:16.257330 12998 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260809 15:10:16.257495 12998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-2/wal
uuid: "aa0cc61b53e84b7994ded5e2e7c0adf4"
format_stamp: "Formatted at 2026-08-09 15:10:16 on dist-test-slave-nj21"
server_key: "504966b60e555af6b46b64b47dfe5cff"
server_key_iv: "860c1aa96db184701fe203a25f537e35"
server_key_version: "kuduclusterkey@0"
I20260809 15:10:16.257722 12998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:10:16.279964 12998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:10:16.283857 12998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:10:16.284142 12998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:10:16.285086 12998 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:10:16.286520 12998 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:10:16.286600 12998 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:16.286724 12998 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:10:16.286777 12998 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:16.322947 12998 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.195:45967
I20260809 15:10:16.322955 13184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.195:45967 every 8 connection(s)
I20260809 15:10:16.324368 12998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
I20260809 15:10:16.328369 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 12998
I20260809 15:10:16.328539 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance
I20260809 15:10:16.329041 25311 external_mini_cluster.cc:1428] Setting key 7a634c9c247f70dc9e414e9e57d476d5
Aug 09 15:10:16 dist-test-slave-nj21 krb5kdc[11695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288216, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:10:16.343854 13189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:16.326885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:39781 (local address 127.24.183.254:34693)
0809 15:10:16.327342 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:16.327346 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:16.328358 (+ 1012us) server_negotiation.cc:407] Connection header received
0809 15:10:16.330345 (+ 1987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:16.330351 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:16.330450 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:16.330545 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:16.332127 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:16.332964 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:16.334502 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:16.334934 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:16.335124 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:16.338259 (+ 3135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:16.338265 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:16.338268 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:16.338302 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:16.340817 (+ 2515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:16.341515 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:16.341519 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:16.341521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:16.341650 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:16.342088 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:16.342092 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:16.342094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:16.342390 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:16.342527 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:16.343245 (+ 718us) server_negotiation.cc:299] Negotiation successful
0809 15:10:16.343462 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":212,"threads_started":1}
I20260809 15:10:16.344759 13191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:16.327471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:34693 (local address 127.24.183.195:39781)
0809 15:10:16.328195 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:16.328239 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:16.330076 (+ 1837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:16.330815 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:16.330827 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:16.331310 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:10:16.331956 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:16.331975 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:16.333123 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:16.333126 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:16.334352 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:16.334364 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:16.335160 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:16.335169 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:16.335423 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:16.336050 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:10:16.336073 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:10:16.338045 (+ 1972us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:10:16.340951 (+ 2906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:16.340960 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:16.340976 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:16.341393 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:16.341754 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:16.341759 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:16.341763 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:16.341971 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:16.342524 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:10:16.342532 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:10:16.342821 (+ 289us) client_negotiation.cc:769] Sending connection context
0809 15:10:16.343109 (+ 288us) client_negotiation.cc:240] Negotiation successful
0809 15:10:16.343560 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":317,"threads_started":1}
I20260809 15:10:16.345933 13187 heartbeater.cc:347] Connected to a master server at 127.24.183.254:34693
I20260809 15:10:16.346216 13187 heartbeater.cc:464] Registering TS with master...
I20260809 15:10:16.346881 13187 heartbeater.cc:511] Master 127.24.183.254:34693 requested a full tablet report, sending...
I20260809 15:10:16.348250 12409 ts_manager.cc:194] Registered new tserver with Master: aa0cc61b53e84b7994ded5e2e7c0adf4 (127.24.183.195:45967)
I20260809 15:10:16.349049 12409 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.195@KRBTEST.COM'} at 127.24.183.195:39781
I20260809 15:10:16.356747 25311 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:10:16.358348 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 12535
Aug 09 15:10:16 dist-test-slave-nj21 krb5kdc[11695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288212, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.193@KRBTEST.COM
I20260809 15:10:16.373974 13193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:16.359197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36384 (local address 127.24.183.193:33831)
0809 15:10:16.359826 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:16.359832 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:16.359856 (+ 24us) server_negotiation.cc:407] Connection header received
0809 15:10:16.359924 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:16.359930 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:16.360098 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:16.360217 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:16.361585 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:16.362584 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:16.364051 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:16.364704 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:16.364817 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:16.368935 (+ 4118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:16.368954 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:16.368966 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:16.369009 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:16.371666 (+ 2657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:16.372325 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:16.372332 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:16.372334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:16.372492 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:16.372790 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:16.372793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:16.372795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:16.373115 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:16.373334 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:16.373504 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 15:10:16.373693 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":315,"threads_started":1}
I20260809 15:10:16.375072 12739 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:10:16.457110 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 12535
I20260809 15:10:16.481359 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 12762
Aug 09 15:10:16 dist-test-slave-nj21 krb5kdc[11695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288212, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.194@KRBTEST.COM
I20260809 15:10:16.497242 13197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:16.482286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34698 (local address 127.24.183.194:43401)
0809 15:10:16.482817 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:16.482824 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:16.482851 (+ 27us) server_negotiation.cc:407] Connection header received
0809 15:10:16.482922 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:16.482928 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:16.483137 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:16.483272 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:16.484678 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:16.485674 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:16.486982 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:16.487514 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:16.487608 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:16.491796 (+ 4188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:16.491808 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:16.491824 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:16.491857 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:16.494625 (+ 2768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:16.495264 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:16.495270 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:16.495273 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:16.495414 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:16.495791 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:16.495794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:16.495796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:16.496147 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:16.496377 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:16.496649 (+ 272us) server_negotiation.cc:299] Negotiation successful
0809 15:10:16.496851 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":251,"threads_started":1}
I20260809 15:10:16.498443 12982 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:10:16.583033 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 12762
I20260809 15:10:16.606945 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 12998
Aug 09 15:10:16 dist-test-slave-nj21 krb5kdc[11695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288212, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.195@KRBTEST.COM
I20260809 15:10:16.622874 13201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:16.607878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41300 (local address 127.24.183.195:45967)
0809 15:10:16.608539 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:16.608546 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:16.608576 (+ 30us) server_negotiation.cc:407] Connection header received
0809 15:10:16.608640 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:16.608646 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:16.608862 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:16.609043 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:16.610323 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:16.611468 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:16.612873 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:16.613458 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:16.613561 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:16.617784 (+ 4223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:16.617796 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:16.617814 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:16.617854 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:16.620645 (+ 2791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:16.621204 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:16.621211 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:16.621213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:16.621344 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:16.621606 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:16.621609 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:16.621611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:16.621947 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:16.622187 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:16.622385 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 15:10:16.622584 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":305,"threads_started":1}
I20260809 15:10:16.624293 13179 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:10:16.706032 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 12998
I20260809 15:10:16.728015 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 12362
I20260809 15:10:16.729259 12458 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:10:16.838541 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 12362
I20260809 15:10:16.868995 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:34693
--webserver_interface=127.24.183.254
--webserver_port=39229
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:43755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:34693
--ranger_config_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.183.212:40483/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.183.212:40483/kms with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:10:17.199102 13205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:10:17.199378 13205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:10:17.199504 13205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:10:17.206651 13205 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:10:17.206759 13205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:10:17.206838 13205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:10:17.206909 13205 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:10:17.206975 13205 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:10:17.216842 13205 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:43755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.183.212:40483/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:34693
--ranger_config_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:34693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=39229
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:10:17.218079 13205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:10:17.219394 13205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:10:17.230576 13213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:17.230644 13214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:17.230674 13216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:10:17.231771 13205 server_base.cc:1034] running on GCE node
I20260809 15:10:17.232458 13205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:10:17.233732 13205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:10:17.234939 13205 hybrid_clock.cc:648] HybridClock initialized: now 1786288217234882 us; error 46 us; skew 500 ppm
I20260809 15:10:17.235275 13205 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:10:17 dist-test-slave-nj21 krb5kdc[11695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288217, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:10:17.238274 13205 init.cc:377] Logged in from keytab as kudu/127.24.183.254@KRBTEST.COM (short username kudu)
I20260809 15:10:17.239652 13205 webserver.cc:460] Webserver started at http://127.24.183.254:39229/ using document root <none> and password file <none>
I20260809 15:10:17.240299 13205 fs_manager.cc:359] Metadata directory not provided
I20260809 15:10:17.240427 13205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 15:10:17 dist-test-slave-nj21 krb5kdc[11695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288217, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for HTTP/127.24.183.212@KRBTEST.COM
I20260809 15:10:17.290172 13205 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.003s
I20260809 15:10:17.293174 13227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:17.294603 13205 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:10:17.294819 13205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "4b471d8ade814391a684ef3b622018e5"
format_stamp: "Formatted at 2026-08-09 15:10:13 on dist-test-slave-nj21"
server_key: "b6ab40ef346807c44f46170bdb5caeb3"
server_key_iv: "96c28a4dc76f26c7bcc2ae62372a350b"
server_key_version: "kuduclusterkey@0"
I20260809 15:10:17.295898 13205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:10:17.366194 13205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:10:17.370301 13205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:10:17.370572 13205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:10:17.388504 13205 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:34693
I20260809 15:10:17.388545 13307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:34693 every 8 connection(s)
I20260809 15:10:17.390030 13205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:10:17.394327 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 13205
I20260809 15:10:17.395206 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.183.212:40483/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.193:33831
--local_ip_for_outbound_sockets=127.24.183.193
--tserver_master_addrs=127.24.183.254:34693
--webserver_port=45413
--webserver_interface=127.24.183.193
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:43755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.183.212:40483/kms with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:10:17.396863 13308 sys_catalog.cc:263] Verifying existing consensus state
I20260809 15:10:17.400135 13308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5: Bootstrap starting.
I20260809 15:10:17.423034 13308 log.cc:824] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5: Log is configured to *not* fsync() on all Append() calls
I20260809 15:10:17.434262 13308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:10:17.434864 13308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5: Bootstrap complete.
I20260809 15:10:17.444180 13308 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b471d8ade814391a684ef3b622018e5" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 34693 } }
I20260809 15:10:17.445017 13308 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b471d8ade814391a684ef3b622018e5, State: Initialized, Role: FOLLOWER
I20260809 15:10:17.445605 13308 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b471d8ade814391a684ef3b622018e5" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 34693 } }
I20260809 15:10:17.445822 13308 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:10:17.445940 13308 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:10:17.446097 13308 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:10:17.449873 13308 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b471d8ade814391a684ef3b622018e5" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 34693 } }
I20260809 15:10:17.450356 13308 leader_election.cc:302] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b471d8ade814391a684ef3b622018e5; no voters:
I20260809 15:10:17.450953 13308 leader_election.cc:288] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 15:10:17.451135 13314 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:10:17.452124 13314 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5 [term 2 LEADER]: Becoming Leader. State: Replica: 4b471d8ade814391a684ef3b622018e5, State: Running, Role: LEADER
I20260809 15:10:17.452837 13308 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:10:17.452723 13314 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b471d8ade814391a684ef3b622018e5" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 34693 } }
I20260809 15:10:17.461421 13315 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4b471d8ade814391a684ef3b622018e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b471d8ade814391a684ef3b622018e5" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 34693 } } }
I20260809 15:10:17.461851 13315 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5 [sys.catalog]: This master's current role is: LEADER
I20260809 15:10:17.462390 13316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b471d8ade814391a684ef3b622018e5. Latest consensus state: current_term: 2 leader_uuid: "4b471d8ade814391a684ef3b622018e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b471d8ade814391a684ef3b622018e5" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 34693 } } }
I20260809 15:10:17.462711 13316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5 [sys.catalog]: This master's current role is: LEADER
I20260809 15:10:17.475212 13327 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:10:17.475696 13308 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 15:10:17.480578 13327 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:10:17.482040 13327 catalog_manager.cc:1259] Loaded cluster ID: 09b1a38cf52f43f4a495c828e5f95bbd
I20260809 15:10:17.482185 13327 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:10:17.493359 13327 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:10:17.495816 13327 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5: Loaded TSK: 0
I20260809 15:10:17.497645 13327 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 15:10:17.863656 13312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:10:17.864144 13312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:10:17.864348 13312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:10:17.875626 13312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:10:17.875854 13312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:10:17.876091 13312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.193
I20260809 15:10:17.891649 13312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:43755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.183.212:40483/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.193:33831
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.183.193
--webserver_port=45413
--tserver_master_addrs=127.24.183.254:34693
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.193
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:10:17.893836 13312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:10:17.895815 13312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:10:17.925026 13369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:17.932061 13366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:10:17.932109 13312 server_base.cc:1034] running on GCE node
W20260809 15:10:17.933117 13372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:10:17.933837 13312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:10:17.934864 13312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:10:17.936066 13312 hybrid_clock.cc:648] HybridClock initialized: now 1786288217935997 us; error 58 us; skew 500 ppm
I20260809 15:10:17.936396 13312 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:10:17 dist-test-slave-nj21 krb5kdc[11695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288217, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:10:17.939606 13312 init.cc:377] Logged in from keytab as kudu/127.24.183.193@KRBTEST.COM (short username kudu)
I20260809 15:10:17.941190 13312 webserver.cc:460] Webserver started at http://127.24.183.193:45413/ using document root <none> and password file <none>
I20260809 15:10:17.941861 13312 fs_manager.cc:359] Metadata directory not provided
I20260809 15:10:17.941994 13312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 15:10:17 dist-test-slave-nj21 krb5kdc[11695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288217, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for HTTP/127.24.183.212@KRBTEST.COM
I20260809 15:10:17.981693 13312 fs_manager.cc:709] Time spent opening directory manager: real 0.010s user 0.007s sys 0.001s
I20260809 15:10:17.991232 13381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:17.993002 13312 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.007s sys 0.000s
I20260809 15:10:17.993165 13312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-0/wal
uuid: "69cbebcd7b864b18a5e8865f5e479bad"
format_stamp: "Formatted at 2026-08-09 15:10:14 on dist-test-slave-nj21"
server_key: "6d04ceae9e8c6fdafdc3a939cb8ea8dc"
server_key_iv: "a1edc2ea949a4a15fdde5b7f6fd6f14d"
server_key_version: "kuduclusterkey@0"
I20260809 15:10:17.994736 13312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:10:18.015928 13312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:10:18.020015 13312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:10:18.020290 13312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:10:18.021332 13312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:10:18.023203 13312 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:10:18.023382 13312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:18.023587 13312 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:10:18.023762 13312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:18.064251 13312 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.193:33831
I20260809 15:10:18.064307 13567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.193:33831 every 8 connection(s)
I20260809 15:10:18.066080 13312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
I20260809 15:10:18.075759 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 13312
I20260809 15:10:18.076632 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.183.212:40483/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.194:43401
--local_ip_for_outbound_sockets=127.24.183.194
--tserver_master_addrs=127.24.183.254:34693
--webserver_port=37147
--webserver_interface=127.24.183.194
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:43755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.183.212:40483/kms with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 15:10:18 dist-test-slave-nj21 krb5kdc[11695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288217, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:10:18.096381 13570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:18.069465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:52517 (local address 127.24.183.254:34693)
0809 15:10:18.075700 (+ 6235us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:18.075718 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:18.075769 (+ 51us) server_negotiation.cc:407] Connection header received
0809 15:10:18.076180 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:18.076201 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:18.076654 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:18.077034 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:18.079306 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:18.081003 (+ 1697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:18.083163 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:18.084119 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:18.084281 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:18.088866 (+ 4585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:18.088877 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:18.088896 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:18.088943 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:18.091869 (+ 2926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:18.092588 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:18.092594 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:18.092600 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:18.092755 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:18.093167 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:18.093172 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:18.093174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:18.093661 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:18.093845 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:18.094613 (+ 768us) server_negotiation.cc:299] Negotiation successful
0809 15:10:18.094989 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5936,"thread_start_us":12909,"threads_started":1}
I20260809 15:10:18.096390 13571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:18.069557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:34693 (local address 127.24.183.193:52517)
0809 15:10:18.073176 (+ 3619us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:18.073231 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:18.074941 (+ 1710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:18.077722 (+ 2781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:18.077732 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:18.078290 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:10:18.079053 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:18.079078 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:18.081198 (+ 2120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:18.081203 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:18.082981 (+ 1778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:18.083007 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:18.084647 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:18.084655 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:18.084937 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:18.085729 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:10:18.085759 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:10:18.088576 (+ 2817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:10:18.092014 (+ 3438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:18.092023 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:18.092036 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:18.092447 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:18.092886 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:18.092890 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:18.092892 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:18.093060 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:18.093819 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:10:18.093827 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:10:18.094130 (+ 303us) client_negotiation.cc:769] Sending connection context
0809 15:10:18.094374 (+ 244us) client_negotiation.cc:240] Negotiation successful
0809 15:10:18.094951 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3335,"thread_start_us":3127,"threads_started":1}
I20260809 15:10:18.099910 13568 heartbeater.cc:347] Connected to a master server at 127.24.183.254:34693
I20260809 15:10:18.100225 13568 heartbeater.cc:464] Registering TS with master...
I20260809 15:10:18.101018 13568 heartbeater.cc:511] Master 127.24.183.254:34693 requested a full tablet report, sending...
W20260809 15:10:18.103873 13568 heartbeater.cc:644] Failed to heartbeat to 127.24.183.254:34693 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 15:10:18.104099 13568 heartbeater.cc:464] Registering TS with master...
I20260809 15:10:18.104346 13568 heartbeater.cc:511] Master 127.24.183.254:34693 requested a full tablet report, sending...
I20260809 15:10:18.105477 13568 heartbeater.cc:464] Registering TS with master...
I20260809 15:10:18.105715 13568 heartbeater.cc:511] Master 127.24.183.254:34693 requested a full tablet report, sending...
W20260809 15:10:18.106688 13568 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 15:10:18.494403 13573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:10:18.494860 13573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:10:18.495028 13573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:10:18.506404 13573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:10:18.506561 13573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:10:18.506771 13573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.194
I20260809 15:10:18.522244 13573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:43755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.183.212:40483/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.194:43401
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.183.194
--webserver_port=37147
--tserver_master_addrs=127.24.183.254:34693
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.194
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:10:18.523955 13573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:10:18.525739 13573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:10:18.543089 13591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:18.543802 13587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:18.544238 13588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:10:18.546367 13573 server_base.cc:1034] running on GCE node
I20260809 15:10:18.547112 13573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:10:18.548377 13573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:10:18.549633 13573 hybrid_clock.cc:648] HybridClock initialized: now 1786288218549567 us; error 48 us; skew 500 ppm
I20260809 15:10:18.550004 13573 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:10:18 dist-test-slave-nj21 krb5kdc[11695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288218, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:10:18.556854 13573 init.cc:377] Logged in from keytab as kudu/127.24.183.194@KRBTEST.COM (short username kudu)
I20260809 15:10:18.558897 13573 webserver.cc:460] Webserver started at http://127.24.183.194:37147/ using document root <none> and password file <none>
I20260809 15:10:18.559891 13573 fs_manager.cc:359] Metadata directory not provided
I20260809 15:10:18.560060 13573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 15:10:18 dist-test-slave-nj21 krb5kdc[11695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288218, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for HTTP/127.24.183.212@KRBTEST.COM
I20260809 15:10:18.589756 13573 fs_manager.cc:709] Time spent opening directory manager: real 0.012s user 0.000s sys 0.011s
I20260809 15:10:18.593490 13602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:18.595268 13573 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260809 15:10:18.595504 13573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-1/wal
uuid: "7c95d6e5bf4e4cf2b4e083d99c636c8c"
format_stamp: "Formatted at 2026-08-09 15:10:15 on dist-test-slave-nj21"
server_key: "3cac3217d5bb2f5e4a76c3532b7cbe7e"
server_key_iv: "c1ad2de230528344a3176e969e256234"
server_key_version: "kuduclusterkey@0"
I20260809 15:10:18.596957 13573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:10:18.623090 13573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:10:18.628252 13573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:10:18.628623 13573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:10:18.636114 13573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:10:18.644474 13573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:10:18.644580 13573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:18.644699 13573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:10:18.644767 13573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:18.711874 13573 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.194:43401
I20260809 15:10:18.711963 13791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.194:43401 every 8 connection(s)
I20260809 15:10:18.714289 13573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
I20260809 15:10:18.721659 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 13573
I20260809 15:10:18.722565 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.183.212:40483/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.195:45967
--local_ip_for_outbound_sockets=127.24.183.195
--tserver_master_addrs=127.24.183.254:34693
--webserver_port=38955
--webserver_interface=127.24.183.195
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:43755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.183.212:40483/kms with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 15:10:18 dist-test-slave-nj21 krb5kdc[11695](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786288218, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 15:10:18 dist-test-slave-nj21 krb5kdc[11695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288218, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:10:18.784613 13797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:18.719785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:59483 (local address 127.24.183.254:34693)
0809 15:10:18.745472 (+ 25687us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:18.745480 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:18.748985 (+ 3505us) server_negotiation.cc:407] Connection header received
0809 15:10:18.749090 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:18.749097 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:18.749206 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:18.749356 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:18.751883 (+ 2527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:18.753190 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:18.755973 (+ 2783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:18.756695 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:18.756824 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:18.771993 (+ 15169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:18.772002 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:18.772007 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:18.772082 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:18.776011 (+ 3929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:18.780880 (+ 4869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:18.780887 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:18.780891 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:18.781063 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:18.781716 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:18.781723 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:18.781727 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:18.782152 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:18.782315 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:18.783905 (+ 1590us) server_negotiation.cc:299] Negotiation successful
0809 15:10:18.784166 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25531,"thread_start_us":25703,"threads_started":1}
I20260809 15:10:18.786917 13796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:18.719103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:34693 (local address 127.24.183.194:59483)
0809 15:10:18.746185 (+ 27082us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:18.746256 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:18.748786 (+ 2530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:18.749919 (+ 1133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:18.749934 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:18.750629 (+ 695us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:10:18.751591 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:18.751623 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:18.753484 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:18.753489 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:18.755761 (+ 2272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:18.755791 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:18.757565 (+ 1774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:18.757576 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:18.757906 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:18.759143 (+ 1237us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:10:18.759180 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:10:18.771561 (+ 12381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:10:18.779844 (+ 8283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:18.779857 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:18.779878 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:18.780708 (+ 830us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:18.781246 (+ 538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:18.781251 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:18.781256 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:18.781556 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:18.782439 (+ 883us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:10:18.782449 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:10:18.782947 (+ 498us) client_negotiation.cc:769] Sending connection context
0809 15:10:18.785016 (+ 2069us) client_negotiation.cc:240] Negotiation successful
0809 15:10:18.785611 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26704,"thread_start_us":26914,"threads_started":1}
I20260809 15:10:18.792580 13793 heartbeater.cc:347] Connected to a master server at 127.24.183.254:34693
I20260809 15:10:18.792905 13793 heartbeater.cc:464] Registering TS with master...
I20260809 15:10:18.793782 13793 heartbeater.cc:511] Master 127.24.183.254:34693 requested a full tablet report, sending...
I20260809 15:10:18.796136 13255 ts_manager.cc:194] Registered new tserver with Master: 7c95d6e5bf4e4cf2b4e083d99c636c8c (127.24.183.194:43401)
I20260809 15:10:18.798518 13255 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.194@KRBTEST.COM'} at 127.24.183.194:59483
I20260809 15:10:19.106864 13568 heartbeater.cc:464] Registering TS with master...
I20260809 15:10:19.107563 13568 heartbeater.cc:511] Master 127.24.183.254:34693 requested a full tablet report, sending...
I20260809 15:10:19.108971 13255 ts_manager.cc:194] Registered new tserver with Master: 69cbebcd7b864b18a5e8865f5e479bad (127.24.183.193:33831)
I20260809 15:10:19.110146 13255 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.193@KRBTEST.COM'} at 127.24.183.193:52517
W20260809 15:10:19.131639 13798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:10:19.131986 13798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:10:19.132134 13798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:10:19.140467 13798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:10:19.140590 13798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:10:19.140743 13798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.195
I20260809 15:10:19.153703 13798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:43755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.183.212:40483/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.195:45967
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.183.195
--webserver_port=38955
--tserver_master_addrs=127.24.183.254:34693
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.195
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:10:19.155169 13798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:10:19.157047 13798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:10:19.178869 13808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:19.179527 13814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:19.187435 13819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:10:19.187988 13798 server_base.cc:1034] running on GCE node
I20260809 15:10:19.188702 13798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:10:19.189798 13798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:10:19.191036 13798 hybrid_clock.cc:648] HybridClock initialized: now 1786288219190957 us; error 49 us; skew 500 ppm
I20260809 15:10:19.191375 13798 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:10:19 dist-test-slave-nj21 krb5kdc[11695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288219, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:10:19.195066 13798 init.cc:377] Logged in from keytab as kudu/127.24.183.195@KRBTEST.COM (short username kudu)
I20260809 15:10:19.196966 13798 webserver.cc:460] Webserver started at http://127.24.183.195:38955/ using document root <none> and password file <none>
I20260809 15:10:19.197696 13798 fs_manager.cc:359] Metadata directory not provided
I20260809 15:10:19.197834 13798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 15:10:19 dist-test-slave-nj21 krb5kdc[11695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288219, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for HTTP/127.24.183.212@KRBTEST.COM
I20260809 15:10:19.236793 13798 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 15:10:19.240937 13829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:19.242795 13798 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260809 15:10:19.242995 13798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-2/wal
uuid: "aa0cc61b53e84b7994ded5e2e7c0adf4"
format_stamp: "Formatted at 2026-08-09 15:10:16 on dist-test-slave-nj21"
server_key: "504966b60e555af6b46b64b47dfe5cff"
server_key_iv: "860c1aa96db184701fe203a25f537e35"
server_key_version: "kuduclusterkey@0"
I20260809 15:10:19.244460 13798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 09 15:10:19 dist-test-slave-nj21 krb5kdc[11695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288218, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for HTTP/127.24.183.212@KRBTEST.COM
I20260809 15:10:19.314985 13798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:10:19.320363 13798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:10:19.320683 13798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:10:19.321988 13798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:10:19.323894 13798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:10:19.324019 13798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:19.324146 13798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:10:19.324229 13798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:19.381119 13798 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.195:45967
I20260809 15:10:19.381183 14013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.195:45967 every 8 connection(s)
I20260809 15:10:19.383124 13798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
I20260809 15:10:19.386447 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 13798
Aug 09 15:10:19 dist-test-slave-nj21 krb5kdc[11695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288219, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:10:19.432709 14019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:19.387504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36132 (local address 127.24.183.254:34693)
0809 15:10:19.414167 (+ 26663us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:19.414173 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:19.414195 (+ 22us) server_negotiation.cc:407] Connection header received
0809 15:10:19.414271 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:19.414277 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:19.414359 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:19.417703 (+ 3344us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:19.417864 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:19.418937 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:19.421012 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:19.421646 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:19.421762 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:19.425279 (+ 3517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:19.425289 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:19.425293 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:19.425332 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:19.428569 (+ 3237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:19.429384 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:19.429389 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:19.429393 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:19.429564 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:19.430258 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:19.430269 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:19.430273 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:19.430631 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:19.430908 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:19.432074 (+ 1166us) server_negotiation.cc:299] Negotiation successful
0809 15:10:19.432323 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26519,"thread_start_us":22988,"threads_started":1}
I20260809 15:10:19.441038 14025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:19.386133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:49393 (local address 127.24.183.254:34693)
0809 15:10:19.410469 (+ 24336us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:19.410476 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:19.415331 (+ 4855us) server_negotiation.cc:407] Connection header received
0809 15:10:19.421938 (+ 6607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:19.421944 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:19.422035 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:19.422113 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:19.424571 (+ 2458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:19.425761 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:19.428237 (+ 2476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:19.428920 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:19.429012 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:19.434059 (+ 5047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:19.434066 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:19.434070 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:19.434114 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:19.437554 (+ 3440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:19.438473 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:19.438478 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:19.438481 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:19.438610 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:19.439133 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:19.439137 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:19.439140 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:19.439447 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:19.439596 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:19.440503 (+ 907us) server_negotiation.cc:299] Negotiation successful
0809 15:10:19.440692 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24184,"thread_start_us":29218,"threads_started":1}
I20260809 15:10:19.442795 14021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:19.386653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:34693 (local address 127.24.183.195:49393)
0809 15:10:19.415146 (+ 28493us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:19.415213 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:19.421595 (+ 6382us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:19.422758 (+ 1163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:19.422773 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:19.423457 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:10:19.424351 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:19.424390 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:19.425966 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:19.425972 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:19.428044 (+ 2072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:19.428071 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:19.429395 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:19.429405 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:19.429718 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:19.430392 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:10:19.430424 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:10:19.433780 (+ 3356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:10:19.437709 (+ 3929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:19.437718 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:19.437734 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:19.438324 (+ 590us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:19.438730 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:19.438735 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:19.438738 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:19.439021 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:19.439601 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:10:19.439610 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:10:19.440076 (+ 466us) client_negotiation.cc:769] Sending connection context
0809 15:10:19.440416 (+ 340us) client_negotiation.cc:240] Negotiation successful
0809 15:10:19.440983 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28133,"thread_start_us":27917,"threads_started":1}
I20260809 15:10:19.444391 14014 heartbeater.cc:347] Connected to a master server at 127.24.183.254:34693
I20260809 15:10:19.444736 14014 heartbeater.cc:464] Registering TS with master...
I20260809 15:10:19.445654 14014 heartbeater.cc:511] Master 127.24.183.254:34693 requested a full tablet report, sending...
I20260809 15:10:19.447305 13255 ts_manager.cc:194] Registered new tserver with Master: aa0cc61b53e84b7994ded5e2e7c0adf4 (127.24.183.195:45967)
I20260809 15:10:19.448477 13255 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.195@KRBTEST.COM'} at 127.24.183.195:49393
I20260809 15:10:19.449126 25311 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:10:19.469400 14025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:19.453719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36138 (local address 127.24.183.254:34693)
0809 15:10:19.453894 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:19.453900 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:19.453969 (+ 69us) server_negotiation.cc:407] Connection header received
0809 15:10:19.454245 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:19.454251 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:19.454356 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:19.454465 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:19.456347 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:19.457403 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:19.459181 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:19.459880 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:19.460009 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:19.463358 (+ 3349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:19.463364 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:19.463368 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:19.463404 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:19.466474 (+ 3070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:19.467261 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:19.467265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:19.467268 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:19.467398 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:19.467870 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:19.467876 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:19.467879 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:19.468216 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:19.468468 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:19.468853 (+ 385us) server_negotiation.cc:299] Negotiation successful
0809 15:10:19.469079 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 15:10:19.475216 13255 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36138:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:10:19.485920 13255 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:10:19.532012 14047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:19.505757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:43401 (local address 127.0.0.1:34712)
0809 15:10:19.515344 (+ 9587us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:19.515382 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:19.515579 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:19.523823 (+ 8244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:19.523836 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:19.523877 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:10:19.524388 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:19.524404 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:19.526357 (+ 1953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:19.526362 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:19.528786 (+ 2424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:19.528805 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:19.530940 (+ 2135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:19.530947 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:19.531162 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:19.531220 (+ 58us) client_negotiation.cc:769] Sending connection context
0809 15:10:19.531366 (+ 146us) client_negotiation.cc:240] Negotiation successful
0809 15:10:19.531588 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9312,"thread_start_us":9194,"threads_started":1}
I20260809 15:10:19.532402 14054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:19.516771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:33831 (local address 127.0.0.1:36390)
0809 15:10:19.524365 (+ 7594us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:19.524384 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:19.524507 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:19.525424 (+ 917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:19.525429 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:19.525453 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:10:19.525863 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:19.525874 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:19.527764 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:19.527770 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:19.529801 (+ 2031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:19.529816 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:19.531649 (+ 1833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:19.531655 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:19.531863 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:19.531893 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 15:10:19.531972 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 15:10:19.532091 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7465,"thread_start_us":8449,"threads_started":1}
I20260809 15:10:19.533032 14048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:19.505988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34712 (local address 127.24.183.194:43401)
0809 15:10:19.516679 (+ 10691us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:19.516687 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:19.516720 (+ 33us) server_negotiation.cc:407] Connection header received
0809 15:10:19.516823 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:19.516831 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:19.517123 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:19.517344 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:10:19.524636 (+ 7292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:19.526172 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:19.528968 (+ 2796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:19.530427 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:19.530595 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:19.530777 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:19.532550 (+ 1773us) server_negotiation.cc:299] Negotiation successful
0809 15:10:19.532725 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10535,"thread_start_us":10412,"threads_started":1}
I20260809 15:10:19.533674 14052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:19.516056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36390 (local address 127.24.183.193:33831)
0809 15:10:19.521575 (+ 5519us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:19.521584 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:19.524629 (+ 3045us) server_negotiation.cc:407] Connection header received
0809 15:10:19.524743 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:19.524750 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:19.524979 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:19.525170 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:10:19.526057 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:19.527586 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:19.530036 (+ 2450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:19.531393 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:19.532850 (+ 1457us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:19.533016 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:19.533240 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 15:10:19.533400 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5339,"thread_start_us":5174,"threads_started":1}
I20260809 15:10:19.537420 13697 tablet_service.cc:1467] Processing CreateTablet for tablet 535a4aa7f74c46e9968bff63f0db398f (DEFAULT_TABLE table=test-table [id=a5c6f26f44e04b819d4fec24863768c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:10:19.537428 13458 tablet_service.cc:1467] Processing CreateTablet for tablet 535a4aa7f74c46e9968bff63f0db398f (DEFAULT_TABLE table=test-table [id=a5c6f26f44e04b819d4fec24863768c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:10:19.538977 13458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 535a4aa7f74c46e9968bff63f0db398f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:10:19.539012 14049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:19.508040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:45967 (local address 127.0.0.1:41316)
0809 15:10:19.523989 (+ 15949us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:19.524013 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:19.524176 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:19.526593 (+ 2417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:19.526598 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:19.526626 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:10:19.527043 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:19.527061 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:19.530481 (+ 3420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:19.530486 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:19.534886 (+ 4400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:19.534902 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:19.538211 (+ 3309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:19.538218 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:19.538395 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:19.538429 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 15:10:19.538515 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 15:10:19.538652 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15802,"mutex_wait_us":3,"thread_start_us":15628,"threads_started":1}
I20260809 15:10:19.539541 13697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 535a4aa7f74c46e9968bff63f0db398f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:10:19.539881 14050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:19.508489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41316 (local address 127.24.183.195:45967)
0809 15:10:19.522867 (+ 14378us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:19.522876 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:19.524335 (+ 1459us) server_negotiation.cc:407] Connection header received
0809 15:10:19.524454 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:19.524463 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:19.524731 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:19.524932 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:10:19.527302 (+ 2370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:19.530292 (+ 2990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:19.535080 (+ 4788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:19.537714 (+ 2634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:19.537902 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:19.538059 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:19.539290 (+ 1231us) server_negotiation.cc:299] Negotiation successful
0809 15:10:19.539476 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14190,"thread_start_us":14015,"threads_started":1}
I20260809 15:10:19.543473 13910 tablet_service.cc:1467] Processing CreateTablet for tablet 535a4aa7f74c46e9968bff63f0db398f (DEFAULT_TABLE table=test-table [id=a5c6f26f44e04b819d4fec24863768c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:10:19.545707 13910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 535a4aa7f74c46e9968bff63f0db398f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:10:19.564345 14057 tablet_bootstrap.cc:492] T 535a4aa7f74c46e9968bff63f0db398f P 69cbebcd7b864b18a5e8865f5e479bad: Bootstrap starting.
I20260809 15:10:19.567965 14057 tablet_bootstrap.cc:654] T 535a4aa7f74c46e9968bff63f0db398f P 69cbebcd7b864b18a5e8865f5e479bad: Neither blocks nor log segments found. Creating new log.
I20260809 15:10:19.569164 14057 log.cc:824] T 535a4aa7f74c46e9968bff63f0db398f P 69cbebcd7b864b18a5e8865f5e479bad: Log is configured to *not* fsync() on all Append() calls
I20260809 15:10:19.576448 14062 tablet_bootstrap.cc:492] T 535a4aa7f74c46e9968bff63f0db398f P 7c95d6e5bf4e4cf2b4e083d99c636c8c: Bootstrap starting.
I20260809 15:10:19.577890 14067 tablet_bootstrap.cc:492] T 535a4aa7f74c46e9968bff63f0db398f P aa0cc61b53e84b7994ded5e2e7c0adf4: Bootstrap starting.
I20260809 15:10:19.580330 14062 tablet_bootstrap.cc:654] T 535a4aa7f74c46e9968bff63f0db398f P 7c95d6e5bf4e4cf2b4e083d99c636c8c: Neither blocks nor log segments found. Creating new log.
I20260809 15:10:19.581670 14062 log.cc:824] T 535a4aa7f74c46e9968bff63f0db398f P 7c95d6e5bf4e4cf2b4e083d99c636c8c: Log is configured to *not* fsync() on all Append() calls
I20260809 15:10:19.582898 14067 tablet_bootstrap.cc:654] T 535a4aa7f74c46e9968bff63f0db398f P aa0cc61b53e84b7994ded5e2e7c0adf4: Neither blocks nor log segments found. Creating new log.
I20260809 15:10:19.584152 14067 log.cc:824] T 535a4aa7f74c46e9968bff63f0db398f P aa0cc61b53e84b7994ded5e2e7c0adf4: Log is configured to *not* fsync() on all Append() calls
I20260809 15:10:19.587443 14057 tablet_bootstrap.cc:492] T 535a4aa7f74c46e9968bff63f0db398f P 69cbebcd7b864b18a5e8865f5e479bad: No bootstrap required, opened a new log
I20260809 15:10:19.588114 14057 ts_tablet_manager.cc:1397] T 535a4aa7f74c46e9968bff63f0db398f P 69cbebcd7b864b18a5e8865f5e479bad: Time spent bootstrapping tablet: real 0.024s user 0.014s sys 0.008s
I20260809 15:10:19.598495 14062 tablet_bootstrap.cc:492] T 535a4aa7f74c46e9968bff63f0db398f P 7c95d6e5bf4e4cf2b4e083d99c636c8c: No bootstrap required, opened a new log
I20260809 15:10:19.598770 14062 ts_tablet_manager.cc:1397] T 535a4aa7f74c46e9968bff63f0db398f P 7c95d6e5bf4e4cf2b4e083d99c636c8c: Time spent bootstrapping tablet: real 0.023s user 0.008s sys 0.000s
I20260809 15:10:19.598807 14057 raft_consensus.cc:348] T 535a4aa7f74c46e9968bff63f0db398f P 69cbebcd7b864b18a5e8865f5e479bad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c95d6e5bf4e4cf2b4e083d99c636c8c" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 43401 } } peers { permanent_uuid: "69cbebcd7b864b18a5e8865f5e479bad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 33831 } } peers { permanent_uuid: "aa0cc61b53e84b7994ded5e2e7c0adf4" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45967 } }
I20260809 15:10:19.599398 14057 raft_consensus.cc:374] T 535a4aa7f74c46e9968bff63f0db398f P 69cbebcd7b864b18a5e8865f5e479bad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:10:19.599514 14057 raft_consensus.cc:729] T 535a4aa7f74c46e9968bff63f0db398f P 69cbebcd7b864b18a5e8865f5e479bad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69cbebcd7b864b18a5e8865f5e479bad, State: Initialized, Role: FOLLOWER
I20260809 15:10:19.600183 14057 consensus_queue.cc:260] T 535a4aa7f74c46e9968bff63f0db398f P 69cbebcd7b864b18a5e8865f5e479bad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c95d6e5bf4e4cf2b4e083d99c636c8c" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 43401 } } peers { permanent_uuid: "69cbebcd7b864b18a5e8865f5e479bad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 33831 } } peers { permanent_uuid: "aa0cc61b53e84b7994ded5e2e7c0adf4" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45967 } }
I20260809 15:10:19.605494 14067 tablet_bootstrap.cc:492] T 535a4aa7f74c46e9968bff63f0db398f P aa0cc61b53e84b7994ded5e2e7c0adf4: No bootstrap required, opened a new log
I20260809 15:10:19.608234 14067 ts_tablet_manager.cc:1397] T 535a4aa7f74c46e9968bff63f0db398f P aa0cc61b53e84b7994ded5e2e7c0adf4: Time spent bootstrapping tablet: real 0.031s user 0.010s sys 0.009s
I20260809 15:10:19.609056 14062 raft_consensus.cc:348] T 535a4aa7f74c46e9968bff63f0db398f P 7c95d6e5bf4e4cf2b4e083d99c636c8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c95d6e5bf4e4cf2b4e083d99c636c8c" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 43401 } } peers { permanent_uuid: "69cbebcd7b864b18a5e8865f5e479bad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 33831 } } peers { permanent_uuid: "aa0cc61b53e84b7994ded5e2e7c0adf4" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45967 } }
I20260809 15:10:19.609506 14062 raft_consensus.cc:374] T 535a4aa7f74c46e9968bff63f0db398f P 7c95d6e5bf4e4cf2b4e083d99c636c8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:10:19.609613 14062 raft_consensus.cc:729] T 535a4aa7f74c46e9968bff63f0db398f P 7c95d6e5bf4e4cf2b4e083d99c636c8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c95d6e5bf4e4cf2b4e083d99c636c8c, State: Initialized, Role: FOLLOWER
I20260809 15:10:19.610355 14062 consensus_queue.cc:260] T 535a4aa7f74c46e9968bff63f0db398f P 7c95d6e5bf4e4cf2b4e083d99c636c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c95d6e5bf4e4cf2b4e083d99c636c8c" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 43401 } } peers { permanent_uuid: "69cbebcd7b864b18a5e8865f5e479bad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 33831 } } peers { permanent_uuid: "aa0cc61b53e84b7994ded5e2e7c0adf4" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45967 } }
I20260809 15:10:19.612931 13568 heartbeater.cc:503] Master 127.24.183.254:34693 was elected leader, sending a full tablet report...
I20260809 15:10:19.613121 14057 ts_tablet_manager.cc:1428] T 535a4aa7f74c46e9968bff63f0db398f P 69cbebcd7b864b18a5e8865f5e479bad: Time spent starting tablet: real 0.025s user 0.021s sys 0.000s
I20260809 15:10:19.614912 14062 ts_tablet_manager.cc:1428] T 535a4aa7f74c46e9968bff63f0db398f P 7c95d6e5bf4e4cf2b4e083d99c636c8c: Time spent starting tablet: real 0.016s user 0.009s sys 0.005s
I20260809 15:10:19.615240 13793 heartbeater.cc:503] Master 127.24.183.254:34693 was elected leader, sending a full tablet report...
I20260809 15:10:19.620136 14067 raft_consensus.cc:348] T 535a4aa7f74c46e9968bff63f0db398f P aa0cc61b53e84b7994ded5e2e7c0adf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c95d6e5bf4e4cf2b4e083d99c636c8c" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 43401 } } peers { permanent_uuid: "69cbebcd7b864b18a5e8865f5e479bad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 33831 } } peers { permanent_uuid: "aa0cc61b53e84b7994ded5e2e7c0adf4" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45967 } }
I20260809 15:10:19.620471 14067 raft_consensus.cc:374] T 535a4aa7f74c46e9968bff63f0db398f P aa0cc61b53e84b7994ded5e2e7c0adf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:10:19.620568 14067 raft_consensus.cc:729] T 535a4aa7f74c46e9968bff63f0db398f P aa0cc61b53e84b7994ded5e2e7c0adf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa0cc61b53e84b7994ded5e2e7c0adf4, State: Initialized, Role: FOLLOWER
I20260809 15:10:19.621206 14067 consensus_queue.cc:260] T 535a4aa7f74c46e9968bff63f0db398f P aa0cc61b53e84b7994ded5e2e7c0adf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c95d6e5bf4e4cf2b4e083d99c636c8c" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 43401 } } peers { permanent_uuid: "69cbebcd7b864b18a5e8865f5e479bad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 33831 } } peers { permanent_uuid: "aa0cc61b53e84b7994ded5e2e7c0adf4" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45967 } }
I20260809 15:10:19.628500 14074 raft_consensus.cc:482] T 535a4aa7f74c46e9968bff63f0db398f P 7c95d6e5bf4e4cf2b4e083d99c636c8c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:10:19.628832 14074 raft_consensus.cc:504] T 535a4aa7f74c46e9968bff63f0db398f P 7c95d6e5bf4e4cf2b4e083d99c636c8c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c95d6e5bf4e4cf2b4e083d99c636c8c" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 43401 } } peers { permanent_uuid: "69cbebcd7b864b18a5e8865f5e479bad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 33831 } } peers { permanent_uuid: "aa0cc61b53e84b7994ded5e2e7c0adf4" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45967 } }
I20260809 15:10:19.638536 14074 leader_election.cc:288] T 535a4aa7f74c46e9968bff63f0db398f P 7c95d6e5bf4e4cf2b4e083d99c636c8c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 69cbebcd7b864b18a5e8865f5e479bad (127.24.183.193:33831), aa0cc61b53e84b7994ded5e2e7c0adf4 (127.24.183.195:45967)
I20260809 15:10:19.645860 14067 ts_tablet_manager.cc:1428] T 535a4aa7f74c46e9968bff63f0db398f P aa0cc61b53e84b7994ded5e2e7c0adf4: Time spent starting tablet: real 0.037s user 0.023s sys 0.005s
I20260809 15:10:19.649115 14014 heartbeater.cc:503] Master 127.24.183.254:34693 was elected leader, sending a full tablet report...
I20260809 15:10:19.650790 14052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:19.632721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:35867 (local address 127.24.183.193:33831)
0809 15:10:19.632940 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:19.632946 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:19.642863 (+ 9917us) server_negotiation.cc:407] Connection header received
0809 15:10:19.643053 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:19.643059 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:19.643158 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:19.643255 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:10:19.644135 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:19.645208 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:19.648004 (+ 2796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:19.649285 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:19.649466 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:19.649561 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:19.650309 (+ 748us) server_negotiation.cc:299] Negotiation successful
0809 15:10:19.650484 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 15:10:19.651811 14077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:19.632677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:33831 (local address 127.24.183.194:35867)
0809 15:10:19.642731 (+ 10054us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:19.642751 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:19.642907 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:19.643448 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:19.643454 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:19.643485 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:10:19.643954 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:19.643966 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:19.645425 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:19.645430 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:19.647804 (+ 2374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:19.647822 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:19.649911 (+ 2089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:19.649918 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:19.650108 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:19.650145 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 15:10:19.651161 (+ 1016us) client_negotiation.cc:240] Negotiation successful
0809 15:10:19.651327 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9898,"thread_start_us":9660,"threads_started":1}
I20260809 15:10:19.653096 13492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "535a4aa7f74c46e9968bff63f0db398f" candidate_uuid: "7c95d6e5bf4e4cf2b4e083d99c636c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69cbebcd7b864b18a5e8865f5e479bad" is_pre_election: true
I20260809 15:10:19.653728 13492 raft_consensus.cc:2454] T 535a4aa7f74c46e9968bff63f0db398f P 69cbebcd7b864b18a5e8865f5e479bad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c95d6e5bf4e4cf2b4e083d99c636c8c in term 0.
I20260809 15:10:19.654507 13608 leader_election.cc:302] T 535a4aa7f74c46e9968bff63f0db398f P 7c95d6e5bf4e4cf2b4e083d99c636c8c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69cbebcd7b864b18a5e8865f5e479bad, 7c95d6e5bf4e4cf2b4e083d99c636c8c; no voters:
I20260809 15:10:19.654954 14074 raft_consensus.cc:2781] T 535a4aa7f74c46e9968bff63f0db398f P 7c95d6e5bf4e4cf2b4e083d99c636c8c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:10:19.655108 14074 raft_consensus.cc:482] T 535a4aa7f74c46e9968bff63f0db398f P 7c95d6e5bf4e4cf2b4e083d99c636c8c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:10:19.655220 14074 raft_consensus.cc:3037] T 535a4aa7f74c46e9968bff63f0db398f P 7c95d6e5bf4e4cf2b4e083d99c636c8c [term 0 FOLLOWER]: Advancing to term 1
Aug 09 15:10:19 dist-test-slave-nj21 krb5kdc[11695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288218, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for HTTP/127.24.183.212@KRBTEST.COM
I20260809 15:10:19.658878 14074 raft_consensus.cc:504] T 535a4aa7f74c46e9968bff63f0db398f P 7c95d6e5bf4e4cf2b4e083d99c636c8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c95d6e5bf4e4cf2b4e083d99c636c8c" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 43401 } } peers { permanent_uuid: "69cbebcd7b864b18a5e8865f5e479bad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 33831 } } peers { permanent_uuid: "aa0cc61b53e84b7994ded5e2e7c0adf4" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45967 } }
I20260809 15:10:19.659499 14074 leader_election.cc:288] T 535a4aa7f74c46e9968bff63f0db398f P 7c95d6e5bf4e4cf2b4e083d99c636c8c [CANDIDATE]: Term 1 election: Requested vote from peers 69cbebcd7b864b18a5e8865f5e479bad (127.24.183.193:33831), aa0cc61b53e84b7994ded5e2e7c0adf4 (127.24.183.195:45967)
I20260809 15:10:19.659952 13492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "535a4aa7f74c46e9968bff63f0db398f" candidate_uuid: "7c95d6e5bf4e4cf2b4e083d99c636c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69cbebcd7b864b18a5e8865f5e479bad"
I20260809 15:10:19.660140 13492 raft_consensus.cc:3037] T 535a4aa7f74c46e9968bff63f0db398f P 69cbebcd7b864b18a5e8865f5e479bad [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:10:19.662673 13492 raft_consensus.cc:2454] T 535a4aa7f74c46e9968bff63f0db398f P 69cbebcd7b864b18a5e8865f5e479bad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c95d6e5bf4e4cf2b4e083d99c636c8c in term 1.
I20260809 15:10:19.663175 13608 leader_election.cc:302] T 535a4aa7f74c46e9968bff63f0db398f P 7c95d6e5bf4e4cf2b4e083d99c636c8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69cbebcd7b864b18a5e8865f5e479bad, 7c95d6e5bf4e4cf2b4e083d99c636c8c; no voters:
I20260809 15:10:19.663534 14074 raft_consensus.cc:2781] T 535a4aa7f74c46e9968bff63f0db398f P 7c95d6e5bf4e4cf2b4e083d99c636c8c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:10:19.665047 14050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:19.638877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:42409 (local address 127.24.183.195:45967)
0809 15:10:19.639074 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:19.639080 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:19.652530 (+ 13450us) server_negotiation.cc:407] Connection header received
0809 15:10:19.652784 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:19.652790 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:19.652866 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:19.652930 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:10:19.653639 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:19.654659 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:19.662059 (+ 7400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:19.663299 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:19.663443 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:19.663519 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:19.664561 (+ 1042us) server_negotiation.cc:299] Negotiation successful
0809 15:10:19.664733 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 15:10:19.665831 14077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:19.638905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:45967 (local address 127.24.183.194:42409)
0809 15:10:19.652437 (+ 13532us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:19.652457 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:19.652628 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:19.653132 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:19.653137 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:19.653163 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:10:19.653392 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:19.653485 (+ 93us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:19.654962 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:19.654966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:19.659352 (+ 4386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:19.659369 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:19.664143 (+ 4774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:19.664150 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:19.664368 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:19.664401 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 15:10:19.665294 (+ 893us) client_negotiation.cc:240] Negotiation successful
0809 15:10:19.665474 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13380,"mutex_wait_us":2256,"thread_start_us":16862,"threads_started":1}
I20260809 15:10:19.666963 13946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "535a4aa7f74c46e9968bff63f0db398f" candidate_uuid: "7c95d6e5bf4e4cf2b4e083d99c636c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa0cc61b53e84b7994ded5e2e7c0adf4" is_pre_election: true
I20260809 15:10:19.667577 13946 raft_consensus.cc:2454] T 535a4aa7f74c46e9968bff63f0db398f P aa0cc61b53e84b7994ded5e2e7c0adf4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c95d6e5bf4e4cf2b4e083d99c636c8c in term 0.
I20260809 15:10:19.668493 13946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "535a4aa7f74c46e9968bff63f0db398f" candidate_uuid: "7c95d6e5bf4e4cf2b4e083d99c636c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa0cc61b53e84b7994ded5e2e7c0adf4"
I20260809 15:10:19.668689 13946 raft_consensus.cc:3037] T 535a4aa7f74c46e9968bff63f0db398f P aa0cc61b53e84b7994ded5e2e7c0adf4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:10:19.672230 13946 raft_consensus.cc:2454] T 535a4aa7f74c46e9968bff63f0db398f P aa0cc61b53e84b7994ded5e2e7c0adf4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c95d6e5bf4e4cf2b4e083d99c636c8c in term 1.
I20260809 15:10:19.690855 14074 raft_consensus.cc:686] T 535a4aa7f74c46e9968bff63f0db398f P 7c95d6e5bf4e4cf2b4e083d99c636c8c [term 1 LEADER]: Becoming Leader. State: Replica: 7c95d6e5bf4e4cf2b4e083d99c636c8c, State: Running, Role: LEADER
I20260809 15:10:19.691495 14074 consensus_queue.cc:237] T 535a4aa7f74c46e9968bff63f0db398f P 7c95d6e5bf4e4cf2b4e083d99c636c8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c95d6e5bf4e4cf2b4e083d99c636c8c" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 43401 } } peers { permanent_uuid: "69cbebcd7b864b18a5e8865f5e479bad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 33831 } } peers { permanent_uuid: "aa0cc61b53e84b7994ded5e2e7c0adf4" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45967 } }
W20260809 15:10:19.726823 13795 tablet.cc:2369] T 535a4aa7f74c46e9968bff63f0db398f P 7c95d6e5bf4e4cf2b4e083d99c636c8c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:10:19.729296 13254 catalog_manager.cc:5495] T 535a4aa7f74c46e9968bff63f0db398f P 7c95d6e5bf4e4cf2b4e083d99c636c8c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c95d6e5bf4e4cf2b4e083d99c636c8c (127.24.183.194). New cstate: current_term: 1 leader_uuid: "7c95d6e5bf4e4cf2b4e083d99c636c8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c95d6e5bf4e4cf2b4e083d99c636c8c" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 43401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "69cbebcd7b864b18a5e8865f5e479bad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 33831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa0cc61b53e84b7994ded5e2e7c0adf4" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45967 } health_report { overall_health: UNKNOWN } } }
I20260809 15:10:19.807284 14048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:19.799269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34718 (local address 127.24.183.194:43401)
0809 15:10:19.799483 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:19.799489 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:19.799510 (+ 21us) server_negotiation.cc:407] Connection header received
0809 15:10:19.799642 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:19.799648 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:19.799780 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:19.799957 (+ 177us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:10:19.800595 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:19.801718 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:19.803949 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:19.804515 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:19.804627 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:19.805376 (+ 749us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:10:19.806127 (+ 751us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:10:19.806272 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:19.806912 (+ 640us) server_negotiation.cc:299] Negotiation successful
0809 15:10:19.807000 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 15:10:19.816371 13492 raft_consensus.cc:1261] T 535a4aa7f74c46e9968bff63f0db398f P 69cbebcd7b864b18a5e8865f5e479bad [term 1 FOLLOWER]: Refusing update from remote peer 7c95d6e5bf4e4cf2b4e083d99c636c8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:10:19.817579 14074 consensus_queue.cc:1035] T 535a4aa7f74c46e9968bff63f0db398f P 7c95d6e5bf4e4cf2b4e083d99c636c8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "69cbebcd7b864b18a5e8865f5e479bad" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 33831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:10:19.819350 13946 raft_consensus.cc:1261] T 535a4aa7f74c46e9968bff63f0db398f P aa0cc61b53e84b7994ded5e2e7c0adf4 [term 1 FOLLOWER]: Refusing update from remote peer 7c95d6e5bf4e4cf2b4e083d99c636c8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260809 15:10:19.819933 13569 tablet.cc:2369] T 535a4aa7f74c46e9968bff63f0db398f P 69cbebcd7b864b18a5e8865f5e479bad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:10:19.820632 14079 consensus_queue.cc:1035] T 535a4aa7f74c46e9968bff63f0db398f P 7c95d6e5bf4e4cf2b4e083d99c636c8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa0cc61b53e84b7994ded5e2e7c0adf4" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 45967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:10:19.853808 14083 mvcc.cc:205] Tried to move back new op lower bound from 7316636548356165632 to 7316636547862462464. Current Snapshot: MvccSnapshot[applied={T|T < 7316636548356165632}]
I20260809 15:10:19.883199 13254 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:10:19.883536 13254 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:10:19.890653 13254 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4b471d8ade814391a684ef3b622018e5: Sending DeleteTablet for 3 replicas of tablet 535a4aa7f74c46e9968bff63f0db398f
I20260809 15:10:19.891880 13910 tablet_service.cc:1514] Processing DeleteTablet for tablet 535a4aa7f74c46e9968bff63f0db398f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:10:19 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:41316
I20260809 15:10:19.893083 13697 tablet_service.cc:1514] Processing DeleteTablet for tablet 535a4aa7f74c46e9968bff63f0db398f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:10:19 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:34712
I20260809 15:10:19.893206 13458 tablet_service.cc:1514] Processing DeleteTablet for tablet 535a4aa7f74c46e9968bff63f0db398f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:10:19 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:36390
I20260809 15:10:19.903198 14102 tablet_replica.cc:330] T 535a4aa7f74c46e9968bff63f0db398f P 7c95d6e5bf4e4cf2b4e083d99c636c8c: stopping tablet replica
I20260809 15:10:19.903653 14102 raft_consensus.cc:2229] T 535a4aa7f74c46e9968bff63f0db398f P 7c95d6e5bf4e4cf2b4e083d99c636c8c [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:10:19.910552 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 13312
I20260809 15:10:19.913455 14102 raft_consensus.cc:2258] T 535a4aa7f74c46e9968bff63f0db398f P 7c95d6e5bf4e4cf2b4e083d99c636c8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:10:19.915532 14102 ts_tablet_manager.cc:1905] T 535a4aa7f74c46e9968bff63f0db398f P 7c95d6e5bf4e4cf2b4e083d99c636c8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:10:19.921526 14102 ts_tablet_manager.cc:1918] T 535a4aa7f74c46e9968bff63f0db398f P 7c95d6e5bf4e4cf2b4e083d99c636c8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:10:19.921694 14102 log.cc:1196] T 535a4aa7f74c46e9968bff63f0db398f P 7c95d6e5bf4e4cf2b4e083d99c636c8c: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-1/wal/wals/535a4aa7f74c46e9968bff63f0db398f
I20260809 15:10:19.922231 14102 ts_tablet_manager.cc:1939] T 535a4aa7f74c46e9968bff63f0db398f P 7c95d6e5bf4e4cf2b4e083d99c636c8c: Deleting consensus metadata
I20260809 15:10:19.927810 14103 tablet_replica.cc:330] T 535a4aa7f74c46e9968bff63f0db398f P 69cbebcd7b864b18a5e8865f5e479bad: stopping tablet replica
I20260809 15:10:19.928297 14103 raft_consensus.cc:2229] T 535a4aa7f74c46e9968bff63f0db398f P 69cbebcd7b864b18a5e8865f5e479bad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:10:19.928843 14103 raft_consensus.cc:2258] T 535a4aa7f74c46e9968bff63f0db398f P 69cbebcd7b864b18a5e8865f5e479bad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:10:19.929823 14100 tablet_replica.cc:330] T 535a4aa7f74c46e9968bff63f0db398f P aa0cc61b53e84b7994ded5e2e7c0adf4: stopping tablet replica
I20260809 15:10:19.929875 13232 catalog_manager.cc:4841] TS 7c95d6e5bf4e4cf2b4e083d99c636c8c (127.24.183.194:43401): tablet 535a4aa7f74c46e9968bff63f0db398f (table test-table [id=a5c6f26f44e04b819d4fec24863768c5]) successfully deleted
I20260809 15:10:19.930352 14100 raft_consensus.cc:2229] T 535a4aa7f74c46e9968bff63f0db398f P aa0cc61b53e84b7994ded5e2e7c0adf4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:10:19.930943 14100 raft_consensus.cc:2258] T 535a4aa7f74c46e9968bff63f0db398f P aa0cc61b53e84b7994ded5e2e7c0adf4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:10:19.932775 14052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:19.912161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36394 (local address 127.24.183.193:33831)
0809 15:10:19.912354 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:19.912360 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:19.912501 (+ 141us) server_negotiation.cc:407] Connection header received
0809 15:10:19.912699 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:19.912705 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:19.912792 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:19.912892 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:19.915099 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:19.916341 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:19.918128 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:19.918721 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:19.918855 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:19.922356 (+ 3501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:19.922377 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:19.922397 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:19.922448 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:19.926727 (+ 4279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:19.927656 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:19.927665 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:19.927689 (+ 24us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:19.927956 (+ 267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:19.928514 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:19.928519 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:19.928523 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:19.929107 (+ 584us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:19.929466 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:19.930976 (+ 1510us) server_negotiation.cc:299] Negotiation successful
0809 15:10:19.932376 (+ 1400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 15:10:19.933133 14103 ts_tablet_manager.cc:1905] T 535a4aa7f74c46e9968bff63f0db398f P 69cbebcd7b864b18a5e8865f5e479bad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:10:19.933754 13561 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:10:19.935196 14100 ts_tablet_manager.cc:1905] T 535a4aa7f74c46e9968bff63f0db398f P aa0cc61b53e84b7994ded5e2e7c0adf4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:10:19.942286 14100 ts_tablet_manager.cc:1918] T 535a4aa7f74c46e9968bff63f0db398f P aa0cc61b53e84b7994ded5e2e7c0adf4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:10:19.942456 14100 log.cc:1196] T 535a4aa7f74c46e9968bff63f0db398f P aa0cc61b53e84b7994ded5e2e7c0adf4: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288021313853-25311-0/minicluster-data/ts-2/wal/wals/535a4aa7f74c46e9968bff63f0db398f
I20260809 15:10:19.942998 14100 ts_tablet_manager.cc:1939] T 535a4aa7f74c46e9968bff63f0db398f P aa0cc61b53e84b7994ded5e2e7c0adf4: Deleting consensus metadata
I20260809 15:10:19.953433 13228 catalog_manager.cc:4841] TS aa0cc61b53e84b7994ded5e2e7c0adf4 (127.24.183.195:45967): tablet 535a4aa7f74c46e9968bff63f0db398f (table test-table [id=a5c6f26f44e04b819d4fec24863768c5]) successfully deleted
I20260809 15:10:20.079993 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 13312
I20260809 15:10:20.110958 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 13573
W20260809 15:10:20.110894 13232 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.183.193:33831 (error 108)
I20260809 15:10:20.113351 14118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:20.112491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:33831 (local address 127.0.0.1:36400)
0809 15:10:20.113071 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:20.113184 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:33831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":345,"threads_started":1}
W20260809 15:10:20.113685 13232 catalog_manager.cc:4584] TS 69cbebcd7b864b18a5e8865f5e479bad (127.24.183.193:33831): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 535a4aa7f74c46e9968bff63f0db398f: Network error: Client connection negotiation failed: client connection to 127.24.183.193:33831: connect: Connection refused (error 111)
I20260809 15:10:20.132386 14048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:20.111813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34734 (local address 127.24.183.194:43401)
0809 15:10:20.112151 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:20.112158 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:20.112181 (+ 23us) server_negotiation.cc:407] Connection header received
0809 15:10:20.112358 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:20.112364 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:20.112451 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:20.112565 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:20.114239 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:20.115082 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:20.116960 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:20.117548 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:20.117659 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:20.126053 (+ 8394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:20.126065 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:20.126081 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:20.126117 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:20.129551 (+ 3434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:20.130255 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:20.130265 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:20.130269 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:20.130444 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:20.130771 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:20.130776 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:20.130779 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:20.131209 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:20.131545 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:20.131747 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 15:10:20.131980 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195}
I20260809 15:10:20.133177 13786 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:10:20.167155 14118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:20.166608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:33831 (local address 127.0.0.1:36408)
0809 15:10:20.166829 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:20.166966 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:33831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260809 15:10:20.168259 14118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:20.167839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:33831 (local address 127.0.0.1:36418)
0809 15:10:20.168010 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:20.168096 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:33831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 15:10:20.228826 14118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:20.228265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:33831 (local address 127.0.0.1:36422)
0809 15:10:20.228509 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:20.228626 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:33831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260809 15:10:20.229949 14118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:20.229524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:33831 (local address 127.0.0.1:36428)
0809 15:10:20.229670 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:20.229763 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:33831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 15:10:20.248569 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 13573
I20260809 15:10:20.273065 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 13798
I20260809 15:10:20.286306 14121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:20.273783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41320 (local address 127.24.183.195:45967)
0809 15:10:20.274263 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:20.274268 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:20.274283 (+ 15us) server_negotiation.cc:407] Connection header received
0809 15:10:20.274336 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:20.274340 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:20.274408 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:20.274500 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:20.275813 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:20.276535 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:20.277837 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:20.278237 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:20.278349 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:20.280840 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:20.280853 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:20.280870 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:20.280903 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:20.283430 (+ 2527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:20.284154 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:20.284162 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:20.284164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:20.284303 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:20.284677 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:20.284681 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:20.284683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:20.285067 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:20.285361 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:20.285693 (+ 332us) server_negotiation.cc:299] Negotiation successful
0809 15:10:20.285930 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":245,"threads_started":1}
I20260809 15:10:20.287002 14006 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:10:20.310251 14118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:20.309671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:33831 (local address 127.0.0.1:36444)
0809 15:10:20.309943 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:20.310055 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:33831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 15:10:20.311352 14118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:20.310884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:33831 (local address 127.0.0.1:36456)
0809 15:10:20.311092 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:20.311178 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:33831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260809 15:10:20.374770 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 13798
I20260809 15:10:20.404104 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 13205
I20260809 15:10:20.405328 13300 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:10:20.519956 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 13205
I20260809 15:10:20.519985 14118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:20.519456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:33831 (local address 127.0.0.1:36462)
0809 15:10:20.519665 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:20.519798 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:33831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 15:10:20.550077 25311 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 15:10:20.893874 25311 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 15:10:20.894104 25311 mini_ranger.cc:66] Stopping Ranger...
I20260809 15:10:21.238051 25311 mini_ranger.cc:68] Stopped Ranger
2026-08-09 15:10:21.238 UTC [11738] LOG: received smart shutdown request
2026-08-09 15:10:21.241 UTC [11738] LOG: background worker "logical replication launcher" (PID 11749) exited with exit code 1
2026-08-09 15:10:21.241 UTC [11743] LOG: shutting down
2026-08-09 15:10:21.383 UTC [11738] LOG: database system is shut down
2026-08-09T15:10:21Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51648 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:10:21 dist-test-slave-nj21 krb5kdc[14140](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:10:21 dist-test-slave-nj21 krb5kdc[14140](info): set up 2 sockets
Aug 09 15:10:21 dist-test-slave-nj21 krb5kdc[14140](info): commencing operation
krb5kdc: starting...
W20260809 15:10:22.712132 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.002s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:10:22 dist-test-slave-nj21 krb5kdc[14140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288222, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:10:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:10:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
I20260809 15:10:22.978760 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:37503
--webserver_interface=127.24.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:42325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:37503,127.24.183.253:33367,127.24.183.252:39827
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:10:23.309544 14161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:10:23.309834 14161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:10:23.309983 14161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:10:23.317178 14161 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:10:23.317287 14161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:10:23.317368 14161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:10:23.317451 14161 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:10:23.317512 14161 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:10:23.327113 14161 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:42325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:37503,127.24.183.253:33367,127.24.183.252:39827
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:37503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:10:23.328423 14161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:10:23.329757 14161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:10:23.341333 14169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:23.341327 14175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:23.341425 14170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:10:23.342481 14161 server_base.cc:1034] running on GCE node
I20260809 15:10:23.343242 14161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:10:23.344667 14161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:10:23.345894 14161 hybrid_clock.cc:648] HybridClock initialized: now 1786288223345828 us; error 47 us; skew 500 ppm
I20260809 15:10:23.346266 14161 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:10:23 dist-test-slave-nj21 krb5kdc[14140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288223, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:10:23.349378 14161 init.cc:377] Logged in from keytab as kudu/127.24.183.254@KRBTEST.COM (short username kudu)
I20260809 15:10:23.350828 14161 webserver.cc:460] Webserver started at http://127.24.183.254:33269/ using document root <none> and password file <none>
I20260809 15:10:23.351511 14161 fs_manager.cc:359] Metadata directory not provided
I20260809 15:10:23.351641 14161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:10:23.351974 14161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:10:23.355041 14161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "332a2583ad1749b0ba5a740501c90c7f"
format_stamp: "Formatted at 2026-08-09 15:10:23 on dist-test-slave-nj21"
server_key: "9e069ab68cf55ef7286145e410f8bcb0"
server_key_iv: "1a8846415fc6e3522d113dde594084d8"
server_key_version: "encryptionkey@0"
I20260809 15:10:23.355976 14161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "332a2583ad1749b0ba5a740501c90c7f"
format_stamp: "Formatted at 2026-08-09 15:10:23 on dist-test-slave-nj21"
server_key: "9e069ab68cf55ef7286145e410f8bcb0"
server_key_iv: "1a8846415fc6e3522d113dde594084d8"
server_key_version: "encryptionkey@0"
I20260809 15:10:23.361164 14161 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 15:10:23.364634 14188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:23.366030 14161 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:10:23.366195 14161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "332a2583ad1749b0ba5a740501c90c7f"
format_stamp: "Formatted at 2026-08-09 15:10:23 on dist-test-slave-nj21"
server_key: "9e069ab68cf55ef7286145e410f8bcb0"
server_key_iv: "1a8846415fc6e3522d113dde594084d8"
server_key_version: "encryptionkey@0"
I20260809 15:10:23.366376 14161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:10:23.385757 14161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:10:23.389922 14161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:10:23.390208 14161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:10:23.408195 14161 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:37503
I20260809 15:10:23.408229 14266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:37503 every 8 connection(s)
I20260809 15:10:23.409540 14161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:10:23.410830 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 14161
I20260809 15:10:23.411083 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:10:23.411957 25311 external_mini_cluster.cc:1428] Setting key b42cb09ca6df74dd024b6fce3ad2969a
I20260809 15:10:23.415493 14267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:10:23.424226 14267 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.183.254" port: 37503 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:10:23 dist-test-slave-nj21 krb5kdc[14140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288223, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.183.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.253@KRBTEST.COM" created.
I20260809 15:10:23.447311 14275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:23.427556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35604 (local address 127.24.183.254:37503)
0809 15:10:23.429269 (+ 1713us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:23.429277 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:23.429331 (+ 54us) server_negotiation.cc:407] Connection header received
0809 15:10:23.430896 (+ 1565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:23.430904 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:23.431183 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:23.431437 (+ 254us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:23.432606 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:23.434362 (+ 1756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:23.435741 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:23.436582 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:23.436776 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:23.440025 (+ 3249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:23.440035 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:23.440051 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:23.440097 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:23.442855 (+ 2758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:23.443578 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:23.443584 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:23.443586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:23.443805 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:23.444217 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:23.444222 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:23.444225 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:23.444749 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:23.444948 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:23.445496 (+ 548us) server_negotiation.cc:299] Negotiation successful
0809 15:10:23.445781 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1502,"spinlock_wait_cycles":43392,"thread_start_us":1282,"threads_started":1}
I20260809 15:10:23.447311 14274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:23.427561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:37503 (local address 127.0.0.1:35604)
0809 15:10:23.428513 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:23.428576 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:23.430533 (+ 1957us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:23.431430 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:23.431436 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:23.432061 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:10:23.432402 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:23.432421 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:23.434500 (+ 2079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:23.434504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:23.435586 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:23.435599 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:23.436815 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:23.436821 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:23.436953 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:23.437556 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:10:23.437579 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:10:23.439775 (+ 2196us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:10:23.442998 (+ 3223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:23.443009 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:23.443018 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:23.443463 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:23.443935 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:23.443938 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:23.443941 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:23.444100 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:23.444881 (+ 781us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:10:23.444892 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:10:23.445105 (+ 213us) client_negotiation.cc:769] Sending connection context
0809 15:10:23.445318 (+ 213us) client_negotiation.cc:240] Negotiation successful
0809 15:10:23.445945 (+ 627us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"mutex_wait_us":148,"thread_start_us":397,"threads_started":1}
I20260809 15:10:23.450402 14267 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.183.253" port: 33367 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:10:23.451496 14274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:23.451123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:33367 (local address 127.0.0.1:57076)
0809 15:10:23.451278 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:23.451381 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.253:33367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 15:10:23.451820 14189 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.183.253:33367: connect: Connection refused (error 111)
I20260809 15:10:23.453269 14274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:23.452958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:33367 (local address 127.0.0.1:57088)
0809 15:10:23.453073 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:23.453138 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.253:33367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 15:10:23.453465 14267 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.253:33367: Network error: Client connection negotiation failed: client connection to 127.24.183.253:33367: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.253.keytab.
Entry for principal kudu/127.24.183.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.253.keytab.
I20260809 15:10:23.475409 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.253:33367
--webserver_interface=127.24.183.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.253.keytab
--principal=kudu/127.24.183.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:42325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:37503,127.24.183.253:33367,127.24.183.252:39827
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:10:23.502867 14267 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.253" port: 33367 } attempt: 1
I20260809 15:10:23.504165 14274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:23.503655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:33367 (local address 127.0.0.1:57104)
0809 15:10:23.503900 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:23.504032 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.253:33367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 15:10:23.505136 14274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:23.504810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:33367 (local address 127.0.0.1:57114)
0809 15:10:23.504935 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:23.505004 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.253:33367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 15:10:23.505314 14267 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.253:33367: Network error: Client connection negotiation failed: client connection to 127.24.183.253:33367: connect: Connection refused (error 111)
I20260809 15:10:23.573529 14267 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.253" port: 33367 } attempt: 2
I20260809 15:10:23.574766 14274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:23.574266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:33367 (local address 127.0.0.1:57116)
0809 15:10:23.574500 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:23.574606 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.253:33367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 15:10:23.575824 14274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:23.575448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:33367 (local address 127.0.0.1:57118)
0809 15:10:23.575585 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:23.575654 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.253:33367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 15:10:23.576025 14267 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.253:33367: Network error: Client connection negotiation failed: client connection to 127.24.183.253:33367: connect: Connection refused (error 111)
I20260809 15:10:23.667245 14267 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.253" port: 33367 } attempt: 3
I20260809 15:10:23.668545 14274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:23.668082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:33367 (local address 127.0.0.1:57124)
0809 15:10:23.668284 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:23.668401 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.253:33367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260809 15:10:23.669682 14274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:23.669279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:33367 (local address 127.0.0.1:57140)
0809 15:10:23.669427 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:23.669515 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.253:33367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 15:10:23.669911 14267 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.253:33367: Network error: Client connection negotiation failed: client connection to 127.24.183.253:33367: connect: Connection refused (error 111)
I20260809 15:10:23.813151 14267 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.253" port: 33367 } attempt: 4
I20260809 15:10:23.814404 14274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:23.813846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:33367 (local address 127.0.0.1:57142)
0809 15:10:23.814166 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:23.814260 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.253:33367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175}
W20260809 15:10:23.815160 14278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
I20260809 15:10:23.815361 14274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:23.815030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:33367 (local address 127.0.0.1:57156)
0809 15:10:23.815149 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:23.815241 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.253:33367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 15:10:23.815464 14278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:10:23.815554 14267 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.253:33367: Network error: Client connection negotiation failed: client connection to 127.24.183.253:33367: connect: Connection refused (error 111)
W20260809 15:10:23.815630 14278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:10:23.823874 14278 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:10:23.823978 14278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:10:23.824025 14278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:10:23.824082 14278 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:10:23.824126 14278 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:10:23.834348 14278 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:42325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:37503,127.24.183.253:33367,127.24.183.252:39827
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.253.keytab
--principal=kudu/127.24.183.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.253:33367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.183.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:10:23.835704 14278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:10:23.837044 14278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:10:23.848369 14291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:23.848507 14293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:23.848479 14296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:10:23.849493 14278 server_base.cc:1034] running on GCE node
I20260809 15:10:23.850286 14278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:10:23.851660 14278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:10:23.852931 14278 hybrid_clock.cc:648] HybridClock initialized: now 1786288223852875 us; error 50 us; skew 500 ppm
I20260809 15:10:23.853267 14278 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:10:23 dist-test-slave-nj21 krb5kdc[14140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288223, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:10:23.859301 14278 init.cc:377] Logged in from keytab as kudu/127.24.183.253@KRBTEST.COM (short username kudu)
I20260809 15:10:23.860752 14278 webserver.cc:460] Webserver started at http://127.24.183.253:34865/ using document root <none> and password file <none>
I20260809 15:10:23.861452 14278 fs_manager.cc:359] Metadata directory not provided
I20260809 15:10:23.861570 14278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:10:23.861864 14278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:10:23.864878 14278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-1/data/instance:
uuid: "af55f2df9c734fa1bbbbba054c2d16d4"
format_stamp: "Formatted at 2026-08-09 15:10:23 on dist-test-slave-nj21"
server_key: "b2ab08e3eb9b92225e99efb842b3703c"
server_key_iv: "edcdfcb4b21ba405b6941e33799c697b"
server_key_version: "encryptionkey@0"
I20260809 15:10:23.865703 14278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-1/wal/instance:
uuid: "af55f2df9c734fa1bbbbba054c2d16d4"
format_stamp: "Formatted at 2026-08-09 15:10:23 on dist-test-slave-nj21"
server_key: "b2ab08e3eb9b92225e99efb842b3703c"
server_key_iv: "edcdfcb4b21ba405b6941e33799c697b"
server_key_version: "encryptionkey@0"
I20260809 15:10:23.870877 14278 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 15:10:23.874223 14307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:23.875507 14278 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 15:10:23.875694 14278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-1/wal
uuid: "af55f2df9c734fa1bbbbba054c2d16d4"
format_stamp: "Formatted at 2026-08-09 15:10:23 on dist-test-slave-nj21"
server_key: "b2ab08e3eb9b92225e99efb842b3703c"
server_key_iv: "edcdfcb4b21ba405b6941e33799c697b"
server_key_version: "encryptionkey@0"
I20260809 15:10:23.875898 14278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:10:23.895859 14278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:10:23.899729 14278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:10:23.900013 14278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:10:23.917850 14278 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.253:33367
I20260809 15:10:23.917901 14391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.253:33367 every 8 connection(s)
I20260809 15:10:23.919188 14278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-1/data/info.pb
I20260809 15:10:23.919893 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 14278
I20260809 15:10:23.920071 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-1/wal/instance
I20260809 15:10:23.920632 25311 external_mini_cluster.cc:1428] Setting key 988122c9c1b1b80874b3c59268995a16
I20260809 15:10:23.923017 14392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:10:23.929536 14392 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.183.254" port: 37503 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:10:23 dist-test-slave-nj21 krb5kdc[14140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288223, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.253@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.183.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.252@KRBTEST.COM" created.
I20260809 15:10:23.949270 14275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:23.932155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35606 (local address 127.24.183.254:37503)
0809 15:10:23.932325 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:23.932330 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:23.933728 (+ 1398us) server_negotiation.cc:407] Connection header received
0809 15:10:23.935591 (+ 1863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:23.935598 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:23.935705 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:23.935807 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:23.937335 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:23.938042 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:23.939544 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:23.940037 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:23.940155 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:23.943488 (+ 3333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:23.943493 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:23.943497 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:23.943533 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:23.946184 (+ 2651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:23.946990 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:23.946995 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:23.946998 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:23.947138 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:23.947601 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:23.947606 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:23.947608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:23.947945 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:23.948059 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:23.948865 (+ 806us) server_negotiation.cc:299] Negotiation successful
0809 15:10:23.949039 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 15:10:23.950762 14396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:23.932801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:37503 (local address 127.0.0.1:35606)
0809 15:10:23.933558 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:23.933603 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:23.935321 (+ 1718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:23.936046 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:23.936054 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:23.936556 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:10:23.937146 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:23.937164 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:23.938200 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:23.938203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:23.939403 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:23.939416 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:23.940258 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:23.940266 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:23.940487 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:23.941153 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:10:23.941176 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:10:23.943265 (+ 2089us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:10:23.946322 (+ 3057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:23.946332 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:23.946348 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:23.946877 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:23.947244 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:23.947249 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:23.947253 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:23.947483 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:23.948088 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:10:23.948096 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:10:23.948513 (+ 417us) client_negotiation.cc:769] Sending connection context
0809 15:10:23.948814 (+ 301us) client_negotiation.cc:240] Negotiation successful
0809 15:10:23.949325 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":313,"threads_started":1}
I20260809 15:10:23.952546 14392 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.183.253" port: 33367 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:10:23 dist-test-slave-nj21 krb5kdc[14140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288223, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.253@KRBTEST.COM for kudu/127.24.183.253@KRBTEST.COM
I20260809 15:10:23.966886 14396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:23.953291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:33367 (local address 127.0.0.1:57158)
0809 15:10:23.953445 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:23.953463 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:23.953556 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:23.954371 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:23.954375 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:23.954772 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:10:23.954942 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:23.954952 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:23.956291 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:23.956296 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:23.957431 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:23.957446 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:23.958488 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:23.958491 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:23.958662 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:23.959154 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:10:23.959170 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:10:23.961055 (+ 1885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:10:23.964674 (+ 3619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:23.964678 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:23.964680 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:23.965017 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:23.965381 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:23.965384 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:23.965386 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:23.965512 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:23.966218 (+ 706us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:10:23.966221 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:10:23.966331 (+ 110us) client_negotiation.cc:769] Sending connection context
0809 15:10:23.966431 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 15:10:23.966632 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 15:10:23.967207 14397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:23.953416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57158 (local address 127.24.183.253:33367)
0809 15:10:23.953986 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:23.953992 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:23.954018 (+ 26us) server_negotiation.cc:407] Connection header received
0809 15:10:23.954059 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:23.954064 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:23.954224 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:23.954331 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:23.955091 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:23.956153 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:23.957576 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:23.958354 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:23.958486 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:23.961272 (+ 2786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:23.961288 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:23.961306 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:23.961345 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:23.964547 (+ 3202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:23.965146 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:23.965154 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:23.965158 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:23.965295 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:23.965605 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:23.965610 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:23.965613 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:23.966109 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:23.966292 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:23.966584 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 15:10:23.966816 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":417,"thread_start_us":272,"threads_started":1}
I20260809 15:10:23.969483 14392 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39827 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:10:23.970706 14396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:23.970290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:39827 (local address 127.0.0.1:43130)
0809 15:10:23.970461 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:23.970571 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:39827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 15:10:23.970960 14316 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.183.252:39827: connect: Connection refused (error 111)
I20260809 15:10:23.972470 14396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:23.972167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:39827 (local address 127.0.0.1:43132)
0809 15:10:23.972306 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:23.972380 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:39827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 15:10:23.972672 14392 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.252:39827: Network error: Client connection negotiation failed: client connection to 127.24.183.252:39827: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.252.keytab.
Entry for principal kudu/127.24.183.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.252.keytab.
I20260809 15:10:23.980865 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.252:39827
--webserver_interface=127.24.183.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.252.keytab
--principal=kudu/127.24.183.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:42325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:37503,127.24.183.253:33367,127.24.183.252:39827
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:10:24.021958 14392 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39827 } attempt: 1
I20260809 15:10:24.023167 14396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:24.022702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:39827 (local address 127.0.0.1:43136)
0809 15:10:24.022900 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:24.023009 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:39827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260809 15:10:24.024191 14396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:24.023854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:39827 (local address 127.0.0.1:43138)
0809 15:10:24.024008 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:24.024077 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:39827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 15:10:24.024391 14392 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.252:39827: Network error: Client connection negotiation failed: client connection to 127.24.183.252:39827: connect: Connection refused (error 111)
I20260809 15:10:24.092617 14392 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39827 } attempt: 2
I20260809 15:10:24.093775 14396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:24.093308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:39827 (local address 127.0.0.1:43144)
0809 15:10:24.093516 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:24.093611 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:39827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260809 15:10:24.094722 14396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:24.094417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:39827 (local address 127.0.0.1:43150)
0809 15:10:24.094533 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:24.094603 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:39827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 15:10:24.094959 14392 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.252:39827: Network error: Client connection negotiation failed: client connection to 127.24.183.252:39827: connect: Connection refused (error 111)
I20260809 15:10:24.114784 14267 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.253" port: 33367 } attempt: 5
Aug 09 15:10:24 dist-test-slave-nj21 krb5kdc[14140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288223, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for kudu/127.24.183.253@KRBTEST.COM
I20260809 15:10:24.128954 14274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:24.115474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:33367 (local address 127.0.0.1:57164)
0809 15:10:24.115665 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:24.115706 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:24.115829 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:24.116164 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:24.116168 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:24.116542 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:10:24.116737 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:24.116746 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:24.118165 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:24.118169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:24.119059 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:24.119072 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:24.120154 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:24.120159 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:24.120299 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:24.120906 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:10:24.120926 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:10:24.122945 (+ 2019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:10:24.126765 (+ 3820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:24.126769 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:24.126772 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:24.127124 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:24.127490 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:24.127493 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:24.127495 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:24.127618 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:24.128219 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:10:24.128222 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:10:24.128355 (+ 133us) client_negotiation.cc:769] Sending connection context
0809 15:10:24.128478 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 15:10:24.128702 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 15:10:24.129154 14397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:24.115578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57164 (local address 127.24.183.253:33367)
0809 15:10:24.115826 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:24.115832 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:24.115857 (+ 25us) server_negotiation.cc:407] Connection header received
0809 15:10:24.115936 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:24.115942 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:24.116028 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:24.116145 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:24.116938 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:24.118022 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:24.119212 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:24.119984 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:24.120128 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:24.123150 (+ 3022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:24.123157 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:24.123162 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:24.123206 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:24.126616 (+ 3410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:24.127237 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:24.127243 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:24.127247 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:24.127398 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:24.127737 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:24.127742 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:24.127745 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:24.128118 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:24.128271 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:24.128610 (+ 339us) server_negotiation.cc:299] Negotiation successful
0809 15:10:24.128836 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 15:10:24.130227 14267 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39827 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:10:24.131345 14274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:24.130950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:39827 (local address 127.0.0.1:43152)
0809 15:10:24.131127 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:24.131213 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:39827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 15:10:24.132359 14274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:24.132021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:39827 (local address 127.0.0.1:43156)
0809 15:10:24.132159 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:24.132246 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:39827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 15:10:24.132566 14267 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.252:39827: Network error: Client connection negotiation failed: client connection to 127.24.183.252:39827: connect: Connection refused (error 111)
I20260809 15:10:24.183800 14267 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39827 } attempt: 1
I20260809 15:10:24.185005 14274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:24.184502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:39827 (local address 127.0.0.1:43170)
0809 15:10:24.184701 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:24.184834 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:39827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 15:10:24.185990 14274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:24.185649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:39827 (local address 127.0.0.1:43186)
0809 15:10:24.185804 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:24.185882 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:39827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 15:10:24.186151 14392 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39827 } attempt: 3
W20260809 15:10:24.186250 14267 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.252:39827: Network error: Client connection negotiation failed: client connection to 127.24.183.252:39827: connect: Connection refused (error 111)
I20260809 15:10:24.187080 14396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:24.186700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:39827 (local address 127.0.0.1:43202)
0809 15:10:24.186861 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:24.186940 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:39827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 15:10:24.188045 14396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:24.187619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:39827 (local address 127.0.0.1:43212)
0809 15:10:24.187857 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:24.187932 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:39827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
W20260809 15:10:24.188244 14392 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.252:39827: Network error: Client connection negotiation failed: client connection to 127.24.183.252:39827: connect: Connection refused (error 111)
I20260809 15:10:24.254470 14267 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39827 } attempt: 2
I20260809 15:10:24.255834 14274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:24.255229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:39827 (local address 127.0.0.1:43228)
0809 15:10:24.255489 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:24.255594 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:39827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260809 15:10:24.256919 14274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:24.256547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:39827 (local address 127.0.0.1:43232)
0809 15:10:24.256719 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:24.256800 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:39827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 15:10:24.257153 14267 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.252:39827: Network error: Client connection negotiation failed: client connection to 127.24.183.252:39827: connect: Connection refused (error 111)
W20260809 15:10:24.314515 14401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:10:24.314805 14401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:10:24.314947 14401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:10:24.322000 14401 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:10:24.322114 14401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:10:24.322194 14401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:10:24.322271 14401 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:10:24.322332 14401 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:10:24.331455 14392 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39827 } attempt: 4
I20260809 15:10:24.331893 14401 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:42325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:37503,127.24.183.253:33367,127.24.183.252:39827
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.252.keytab
--principal=kudu/127.24.183.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.252:39827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.183.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:10:24.332739 14396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:24.332215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:39827 (local address 127.0.0.1:43244)
0809 15:10:24.332488 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:24.332579 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:39827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260809 15:10:24.333431 14401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:10:24.333770 14396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:24.333430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:39827 (local address 127.0.0.1:43260)
0809 15:10:24.333587 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:24.333662 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:39827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 15:10:24.333976 14392 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.252:39827: Network error: Client connection negotiation failed: client connection to 127.24.183.252:39827: connect: Connection refused (error 111)
I20260809 15:10:24.334801 14401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:10:24.346182 14407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:24.346220 14411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:24.346253 14408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:10:24.347404 14401 server_base.cc:1034] running on GCE node
I20260809 15:10:24.348152 14401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:10:24.349414 14401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:10:24.350626 14401 hybrid_clock.cc:648] HybridClock initialized: now 1786288224350570 us; error 44 us; skew 500 ppm
I20260809 15:10:24.350955 14401 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:10:24 dist-test-slave-nj21 krb5kdc[14140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288224, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:10:24.353871 14401 init.cc:377] Logged in from keytab as kudu/127.24.183.252@KRBTEST.COM (short username kudu)
I20260809 15:10:24.355242 14401 webserver.cc:460] Webserver started at http://127.24.183.252:33897/ using document root <none> and password file <none>
I20260809 15:10:24.355950 14401 fs_manager.cc:359] Metadata directory not provided
I20260809 15:10:24.356067 14401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:10:24.356346 14401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:10:24.359326 14401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-2/data/instance:
uuid: "29e27b15af2b46f2882033ac1b6f5e79"
format_stamp: "Formatted at 2026-08-09 15:10:24 on dist-test-slave-nj21"
server_key: "b9e5fd3d71d095097c26dab0b634b153"
server_key_iv: "7c6410bc5dbc983160f74b1e6a80cdec"
server_key_version: "encryptionkey@0"
I20260809 15:10:24.360138 14401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-2/wal/instance:
uuid: "29e27b15af2b46f2882033ac1b6f5e79"
format_stamp: "Formatted at 2026-08-09 15:10:24 on dist-test-slave-nj21"
server_key: "b9e5fd3d71d095097c26dab0b634b153"
server_key_iv: "7c6410bc5dbc983160f74b1e6a80cdec"
server_key_version: "encryptionkey@0"
I20260809 15:10:24.363361 14267 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39827 } attempt: 3
I20260809 15:10:24.364377 14274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:24.363930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:39827 (local address 127.0.0.1:43262)
0809 15:10:24.364118 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:24.364199 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:39827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 15:10:24.365375 14274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:24.365015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:39827 (local address 127.0.0.1:43264)
0809 15:10:24.365199 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:24.365272 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:39827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260809 15:10:24.365334 14401 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
W20260809 15:10:24.365583 14267 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.252:39827: Network error: Client connection negotiation failed: client connection to 127.24.183.252:39827: connect: Connection refused (error 111)
I20260809 15:10:24.368597 14423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:24.369815 14401 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:10:24.369968 14401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-2/wal
uuid: "29e27b15af2b46f2882033ac1b6f5e79"
format_stamp: "Formatted at 2026-08-09 15:10:24 on dist-test-slave-nj21"
server_key: "b9e5fd3d71d095097c26dab0b634b153"
server_key_iv: "7c6410bc5dbc983160f74b1e6a80cdec"
server_key_version: "encryptionkey@0"
I20260809 15:10:24.370129 14401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:10:24.391016 14401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:10:24.395133 14401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:10:24.395416 14401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:10:24.413754 14401 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.252:39827
I20260809 15:10:24.413756 14510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.252:39827 every 8 connection(s)
I20260809 15:10:24.415375 14401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-2/data/info.pb
I20260809 15:10:24.419155 14511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:10:24.424435 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 14401
I20260809 15:10:24.424608 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288021313853-25311-0/minicluster-data/master-2/wal/instance
I20260809 15:10:24.425038 25311 external_mini_cluster.cc:1428] Setting key 93cfd7175bfabf23560cf09a9c1e9b79
I20260809 15:10:24.425488 14511 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.183.254" port: 37503 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:10:24 dist-test-slave-nj21 krb5kdc[14140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288224, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.252@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
Aug 09 15:10:24 dist-test-slave-nj21 krb5kdc[14140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288222, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.252@KRBTEST.COM
I20260809 15:10:24.445251 14275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:24.428247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35610 (local address 127.24.183.254:37503)
0809 15:10:24.428436 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:24.428443 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:24.429253 (+ 810us) server_negotiation.cc:407] Connection header received
0809 15:10:24.429952 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:24.429959 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:24.430039 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:24.430129 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:24.431794 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:24.432824 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:24.434702 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:24.435127 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:24.435252 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:24.439189 (+ 3937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:24.439196 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:24.439201 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:24.439242 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:24.442022 (+ 2780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:24.442917 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:24.442920 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:24.442923 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:24.443045 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:24.443506 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:24.443508 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:24.443510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:24.443798 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:24.443965 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:24.444654 (+ 689us) server_negotiation.cc:299] Negotiation successful
0809 15:10:24.444881 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 15:10:24.446254 14518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:24.428267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:37503 (local address 127.0.0.1:35610)
0809 15:10:24.429104 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:24.429151 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:24.429697 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:24.430206 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:24.430212 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:24.431051 (+ 839us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:10:24.431604 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:24.431629 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:24.432996 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:24.433000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:24.434562 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:24.434577 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:24.435342 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:24.435347 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:24.435514 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:24.436253 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:10:24.436283 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:10:24.438998 (+ 2715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:10:24.442146 (+ 3148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:24.442158 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:24.442169 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:24.442809 (+ 640us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:24.443151 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:24.443156 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:24.443160 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:24.443410 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:24.443909 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:10:24.443916 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:10:24.444280 (+ 364us) client_negotiation.cc:769] Sending connection context
0809 15:10:24.444540 (+ 260us) client_negotiation.cc:240] Negotiation successful
0809 15:10:24.444983 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":405,"threads_started":1}
I20260809 15:10:24.446712 14517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:24.426858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43266 (local address 127.24.183.252:39827)
0809 15:10:24.427653 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:24.427662 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:24.427733 (+ 71us) server_negotiation.cc:407] Connection header received
0809 15:10:24.429685 (+ 1952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:24.429707 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:24.429949 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:24.430151 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:24.432004 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:24.433468 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:24.435200 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:24.435834 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:24.435950 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:24.440520 (+ 4570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:24.440528 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:24.440539 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:24.440573 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:24.443688 (+ 3115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:24.444415 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:24.444422 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:24.444424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:24.444557 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:24.444929 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:24.444933 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:24.444935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:24.445260 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:24.445533 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:24.446056 (+ 523us) server_negotiation.cc:299] Negotiation successful
0809 15:10:24.446321 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":362,"threads_started":1}
I20260809 15:10:24.447876 14511 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.183.253" port: 33367 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:10:24 dist-test-slave-nj21 krb5kdc[14140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288224, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.252@KRBTEST.COM for kudu/127.24.183.253@KRBTEST.COM
Aug 09 15:10:24 dist-test-slave-nj21 krb5kdc[14140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288222, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:10:24.461257 14518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:24.448552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:33367 (local address 127.0.0.1:57172)
0809 15:10:24.448726 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:24.448742 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:24.448816 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:24.449179 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:24.449184 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:24.449803 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:10:24.450002 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:24.450014 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:24.450983 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:24.450988 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:24.452355 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:24.452370 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:24.453284 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:24.453288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:24.453439 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:24.453910 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:10:24.453927 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:10:24.456083 (+ 2156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:10:24.458964 (+ 2881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:24.458969 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:24.458973 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:24.459454 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:24.459920 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:24.459923 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:24.459926 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:24.460066 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:24.460601 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:10:24.460604 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:10:24.460727 (+ 123us) client_negotiation.cc:769] Sending connection context
0809 15:10:24.460830 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 15:10:24.461011 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 15:10:24.461480 14397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:24.448587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57172 (local address 127.24.183.253:33367)
0809 15:10:24.448775 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:24.448780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:24.448796 (+ 16us) server_negotiation.cc:407] Connection header received
0809 15:10:24.448949 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:24.448954 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:24.449039 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:24.449141 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:24.450155 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:24.450850 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:24.452501 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:24.453159 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:24.453283 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:24.456295 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:24.456303 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:24.456308 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:24.456352 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:24.458843 (+ 2491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:24.459550 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:24.459553 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:24.459556 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:24.459811 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:24.460163 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:24.460168 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:24.460171 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:24.460480 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:24.460629 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:24.460944 (+ 315us) server_negotiation.cc:299] Negotiation successful
0809 15:10:24.461156 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 15:10:24.462399 14511 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39827 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:10:24.465361 14275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:24.449672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35612 (local address 127.24.183.254:37503)
0809 15:10:24.449813 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:24.449817 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:24.449831 (+ 14us) server_negotiation.cc:407] Connection header received
0809 15:10:24.449973 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:24.449977 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:24.450026 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:24.450094 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:24.452544 (+ 2450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:24.453550 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:24.454748 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:24.455368 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:24.455483 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:24.459717 (+ 4234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:24.459725 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:24.459730 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:24.459768 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:24.462950 (+ 3182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:24.463571 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:24.463575 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:24.463577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:24.463728 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:24.464097 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:24.464102 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:24.464105 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:24.464447 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:24.464668 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:24.464994 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 15:10:24.465147 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
Aug 09 15:10:24 dist-test-slave-nj21 krb5kdc[14140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288224, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.252@KRBTEST.COM for kudu/127.24.183.252@KRBTEST.COM
Aug 09 15:10:24 dist-test-slave-nj21 krb5kdc[14140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288222, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.253@KRBTEST.COM
I20260809 15:10:24.474746 14518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:24.463161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:39827 (local address 127.0.0.1:43272)
0809 15:10:24.463301 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:24.463318 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:24.463414 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:24.463794 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:24.463799 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:24.464167 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:10:24.464341 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:24.464352 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:24.465594 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:24.465600 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:24.466743 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:24.466758 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:24.467451 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:24.467457 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:24.467641 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:24.468137 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:10:24.468152 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:10:24.469934 (+ 1782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:10:24.472677 (+ 2743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:24.472681 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:24.472685 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:24.473019 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:24.473360 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:24.473364 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:24.473367 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:24.473519 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:24.474002 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:10:24.474006 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:10:24.474131 (+ 125us) client_negotiation.cc:769] Sending connection context
0809 15:10:24.474243 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 15:10:24.474497 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 15:10:24.475215 14517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:24.463369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43272 (local address 127.24.183.252:39827)
0809 15:10:24.463537 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:24.463543 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:24.463559 (+ 16us) server_negotiation.cc:407] Connection header received
0809 15:10:24.463601 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:24.463606 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:24.463661 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:24.463764 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:24.464496 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:24.465455 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:24.466899 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:24.467318 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:24.467423 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:24.470096 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:24.470102 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:24.470106 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:24.470133 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:24.472565 (+ 2432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:24.473117 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:24.473123 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:24.473126 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:24.473261 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:24.473601 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:24.473606 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:24.473609 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:24.473886 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:24.474031 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:24.474511 (+ 480us) server_negotiation.cc:299] Negotiation successful
0809 15:10:24.474784 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 15:10:24.481222 14511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29e27b15af2b46f2882033ac1b6f5e79: Bootstrap starting.
I20260809 15:10:24.481331 14397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:24.466906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57186 (local address 127.24.183.253:33367)
0809 15:10:24.467056 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:24.467061 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:24.467079 (+ 18us) server_negotiation.cc:407] Connection header received
0809 15:10:24.467224 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:24.467229 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:24.467290 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:24.467366 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:24.468560 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:24.469343 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:24.470634 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:24.471065 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:24.471169 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:24.475467 (+ 4298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:24.475471 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:24.475475 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:24.475504 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:24.478502 (+ 2998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:24.479206 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:24.479211 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:24.479214 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:24.479372 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:24.479857 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:24.479862 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:24.479865 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:24.480228 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:24.480502 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:24.480791 (+ 289us) server_negotiation.cc:299] Negotiation successful
0809 15:10:24.481009 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 15:10:24.484944 14511 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29e27b15af2b46f2882033ac1b6f5e79: Neither blocks nor log segments found. Creating new log.
I20260809 15:10:24.486152 25311 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:10:24.486194 14511 log.cc:824] T 00000000000000000000000000000000 P 29e27b15af2b46f2882033ac1b6f5e79: Log is configured to *not* fsync() on all Append() calls
I20260809 15:10:24.488857 14511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29e27b15af2b46f2882033ac1b6f5e79: No bootstrap required, opened a new log
I20260809 15:10:24.499279 14511 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29e27b15af2b46f2882033ac1b6f5e79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332a2583ad1749b0ba5a740501c90c7f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 37503 } } peers { permanent_uuid: "af55f2df9c734fa1bbbbba054c2d16d4" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 33367 } } peers { permanent_uuid: "29e27b15af2b46f2882033ac1b6f5e79" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39827 } }
I20260809 15:10:24.499725 14511 raft_consensus.cc:374] T 00000000000000000000000000000000 P 29e27b15af2b46f2882033ac1b6f5e79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:10:24.499846 14511 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29e27b15af2b46f2882033ac1b6f5e79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29e27b15af2b46f2882033ac1b6f5e79, State: Initialized, Role: FOLLOWER
I20260809 15:10:24.500609 14511 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29e27b15af2b46f2882033ac1b6f5e79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332a2583ad1749b0ba5a740501c90c7f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 37503 } } peers { permanent_uuid: "af55f2df9c734fa1bbbbba054c2d16d4" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 33367 } } peers { permanent_uuid: "29e27b15af2b46f2882033ac1b6f5e79" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39827 } }
I20260809 15:10:24.507597 14527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29e27b15af2b46f2882033ac1b6f5e79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332a2583ad1749b0ba5a740501c90c7f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 37503 } } peers { permanent_uuid: "af55f2df9c734fa1bbbbba054c2d16d4" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 33367 } } peers { permanent_uuid: "29e27b15af2b46f2882033ac1b6f5e79" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39827 } } }
I20260809 15:10:24.508016 14527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29e27b15af2b46f2882033ac1b6f5e79 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:10:24.509881 14511 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29e27b15af2b46f2882033ac1b6f5e79 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:10:24.523078 14275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:24.491393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35620 (local address 127.24.183.254:37503)
0809 15:10:24.491578 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:24.491585 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:24.501893 (+ 10308us) server_negotiation.cc:407] Connection header received
0809 15:10:24.502375 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:24.502394 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:24.502489 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:24.502569 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:24.505988 (+ 3419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:24.507155 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:24.509628 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:24.510246 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:24.510357 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:24.514675 (+ 4318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:24.514684 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:24.514689 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:24.514732 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:24.519183 (+ 4451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:24.520624 (+ 1441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:24.520629 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:24.520633 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:24.520810 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:24.521294 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:24.521299 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:24.521302 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:24.521678 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:24.521911 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:24.522472 (+ 561us) server_negotiation.cc:299] Negotiation successful
0809 15:10:24.522714 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 15:10:24.523772 14517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:24.491051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43288 (local address 127.24.183.252:39827)
0809 15:10:24.495845 (+ 4794us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:24.495851 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:24.506060 (+ 10209us) server_negotiation.cc:407] Connection header received
0809 15:10:24.506315 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:24.506320 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:24.506394 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:24.506476 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:24.508727 (+ 2251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:24.509737 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:24.511902 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:24.512632 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:24.512727 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:24.516550 (+ 3823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:24.516557 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:24.516562 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:24.516600 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:24.519740 (+ 3140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:24.520697 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:24.520703 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:24.520706 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:24.520965 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:24.521521 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:24.521526 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:24.521529 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:24.521913 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:24.522171 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:24.523164 (+ 993us) server_negotiation.cc:299] Negotiation successful
0809 15:10:24.523366 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4600}
I20260809 15:10:24.523877 14397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:24.491472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57194 (local address 127.24.183.253:33367)
0809 15:10:24.497209 (+ 5737us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:24.497215 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:24.505646 (+ 8431us) server_negotiation.cc:407] Connection header received
0809 15:10:24.505959 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:24.505965 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:24.506053 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:24.506146 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:24.508460 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:24.509506 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:24.511616 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:24.512317 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:24.512410 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:24.516204 (+ 3794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:24.516211 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:24.516216 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:24.516261 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:24.519748 (+ 3487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:24.521487 (+ 1739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:24.521493 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:24.521496 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:24.521650 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:24.522120 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:24.522125 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:24.522128 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:24.522499 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:24.522716 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:24.523284 (+ 568us) server_negotiation.cc:299] Negotiation successful
0809 15:10:24.523517 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5614}
W20260809 15:10:24.534508 14548 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 29e27b15af2b46f2882033ac1b6f5e79: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:10:24.534643 14548 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:10:24.528849 14401 master_runner.cc:186] Error getting master registration for 127.24.183.254:37503: OK, instance_id { permanent_uuid: "332a2583ad1749b0ba5a740501c90c7f" instance_seqno: 1786288223403507 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 15:10:24.538573 14401 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 15:10:24.542775 14267 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39827 } attempt: 4
Aug 09 15:10:24 dist-test-slave-nj21 krb5kdc[14140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288223, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for kudu/127.24.183.252@KRBTEST.COM
I20260809 15:10:24.557451 14274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:24.543273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:39827 (local address 127.0.0.1:43290)
0809 15:10:24.543449 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:24.543466 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:24.543551 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:24.543831 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:24.543834 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:24.544192 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:10:24.544372 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:24.544381 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:24.545772 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:24.545776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:24.546679 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:24.546690 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:24.547703 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:24.547710 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:24.547839 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:24.548421 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:10:24.548445 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:10:24.550897 (+ 2452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:10:24.554664 (+ 3767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:24.554668 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:24.554671 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:24.555067 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:24.555463 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:24.555467 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:24.555471 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:24.555643 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:24.556438 (+ 795us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:10:24.556443 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:10:24.556622 (+ 179us) client_negotiation.cc:769] Sending connection context
0809 15:10:24.556797 (+ 175us) client_negotiation.cc:240] Negotiation successful
0809 15:10:24.557067 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 15:10:24.557467 14517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:24.543321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43290 (local address 127.24.183.252:39827)
0809 15:10:24.543545 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:24.543550 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:24.543573 (+ 23us) server_negotiation.cc:407] Connection header received
0809 15:10:24.543624 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:24.543629 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:24.543724 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:24.543820 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:24.544506 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:24.545668 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:24.546814 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:24.547544 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:24.547694 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:24.551082 (+ 3388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:24.551089 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:24.551094 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:24.551141 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:24.554526 (+ 3385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:24.555176 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:24.555181 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:24.555185 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:24.555343 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:24.555829 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:24.555835 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:24.555838 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:24.556238 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:24.556372 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:24.556898 (+ 526us) server_negotiation.cc:299] Negotiation successful
0809 15:10:24.557117 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260809 15:10:24.561949 14267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 332a2583ad1749b0ba5a740501c90c7f: Bootstrap starting.
I20260809 15:10:24.565250 14267 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 332a2583ad1749b0ba5a740501c90c7f: Neither blocks nor log segments found. Creating new log.
I20260809 15:10:24.566421 14267 log.cc:824] T 00000000000000000000000000000000 P 332a2583ad1749b0ba5a740501c90c7f: Log is configured to *not* fsync() on all Append() calls
I20260809 15:10:24.569365 14267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 332a2583ad1749b0ba5a740501c90c7f: No bootstrap required, opened a new log
I20260809 15:10:24.576481 14267 raft_consensus.cc:348] T 00000000000000000000000000000000 P 332a2583ad1749b0ba5a740501c90c7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332a2583ad1749b0ba5a740501c90c7f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 37503 } } peers { permanent_uuid: "af55f2df9c734fa1bbbbba054c2d16d4" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 33367 } } peers { permanent_uuid: "29e27b15af2b46f2882033ac1b6f5e79" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39827 } }
I20260809 15:10:24.576776 14267 raft_consensus.cc:374] T 00000000000000000000000000000000 P 332a2583ad1749b0ba5a740501c90c7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:10:24.576892 14267 raft_consensus.cc:729] T 00000000000000000000000000000000 P 332a2583ad1749b0ba5a740501c90c7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 332a2583ad1749b0ba5a740501c90c7f, State: Initialized, Role: FOLLOWER
I20260809 15:10:24.577409 14267 consensus_queue.cc:260] T 00000000000000000000000000000000 P 332a2583ad1749b0ba5a740501c90c7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332a2583ad1749b0ba5a740501c90c7f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 37503 } } peers { permanent_uuid: "af55f2df9c734fa1bbbbba054c2d16d4" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 33367 } } peers { permanent_uuid: "29e27b15af2b46f2882033ac1b6f5e79" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39827 } }
I20260809 15:10:24.578326 14553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 332a2583ad1749b0ba5a740501c90c7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332a2583ad1749b0ba5a740501c90c7f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 37503 } } peers { permanent_uuid: "af55f2df9c734fa1bbbbba054c2d16d4" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 33367 } } peers { permanent_uuid: "29e27b15af2b46f2882033ac1b6f5e79" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39827 } } }
I20260809 15:10:24.578730 14553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 332a2583ad1749b0ba5a740501c90c7f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:10:24.579720 14267 sys_catalog.cc:564] T 00000000000000000000000000000000 P 332a2583ad1749b0ba5a740501c90c7f [sys.catalog]: configured and running, proceeding with master startup.
W20260809 15:10:24.589250 14574 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 332a2583ad1749b0ba5a740501c90c7f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:10:24.589491 14574 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:10:24.588564 14161 master_runner.cc:186] Error getting master registration for 127.24.183.253:33367: OK, instance_id { permanent_uuid: "af55f2df9c734fa1bbbbba054c2d16d4" instance_seqno: 1786288223912657 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 15:10:24.597028 14161 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 15:10:24.633242 14392 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39827 } attempt: 5
Aug 09 15:10:24 dist-test-slave-nj21 krb5kdc[14140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288223, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.253@KRBTEST.COM for kudu/127.24.183.252@KRBTEST.COM
W20260809 15:10:24.643060 14197 tablet.cc:2369] T 00000000000000000000000000000000 P 332a2583ad1749b0ba5a740501c90c7f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:10:24.647627 14435 tablet.cc:2369] T 00000000000000000000000000000000 P 29e27b15af2b46f2882033ac1b6f5e79: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:10:24.648777 14396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:24.633859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:39827 (local address 127.0.0.1:43302)
0809 15:10:24.634157 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:24.634177 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:24.634305 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:24.634631 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:24.634637 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:24.635092 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:10:24.635310 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:24.635324 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:24.637785 (+ 2461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:24.637791 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:24.638694 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:24.638707 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:24.639454 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:24.639460 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:24.639648 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:24.640237 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:10:24.640264 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:10:24.642292 (+ 2028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:10:24.645810 (+ 3518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:24.645816 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:24.645821 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:24.646190 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:24.646650 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:24.646656 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:24.646661 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:24.646855 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:24.647528 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:10:24.647533 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:10:24.647736 (+ 203us) client_negotiation.cc:769] Sending connection context
0809 15:10:24.648026 (+ 290us) client_negotiation.cc:240] Negotiation successful
0809 15:10:24.648369 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164}
I20260809 15:10:24.649914 14517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:24.633958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43302 (local address 127.24.183.252:39827)
0809 15:10:24.634170 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:24.634178 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:24.634232 (+ 54us) server_negotiation.cc:407] Connection header received
0809 15:10:24.634428 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:24.634432 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:24.634496 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:24.634585 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:24.635606 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:24.636775 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:24.638838 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:24.639304 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:24.639414 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:24.642494 (+ 3080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:24.642499 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:24.642502 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:24.642536 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:24.645680 (+ 3144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:24.646312 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:24.646317 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:24.646319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:24.646459 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:24.646988 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:24.646992 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:24.646994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:24.647302 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:24.647456 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:24.649270 (+ 1814us) server_negotiation.cc:299] Negotiation successful
0809 15:10:24.649561 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260809 15:10:24.655360 14392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af55f2df9c734fa1bbbbba054c2d16d4: Bootstrap starting.
I20260809 15:10:24.658524 14392 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af55f2df9c734fa1bbbbba054c2d16d4: Neither blocks nor log segments found. Creating new log.
I20260809 15:10:24.660287 14392 log.cc:824] T 00000000000000000000000000000000 P af55f2df9c734fa1bbbbba054c2d16d4: Log is configured to *not* fsync() on all Append() calls
I20260809 15:10:24.662953 14392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af55f2df9c734fa1bbbbba054c2d16d4: No bootstrap required, opened a new log
I20260809 15:10:24.670116 14392 raft_consensus.cc:348] T 00000000000000000000000000000000 P af55f2df9c734fa1bbbbba054c2d16d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332a2583ad1749b0ba5a740501c90c7f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 37503 } } peers { permanent_uuid: "af55f2df9c734fa1bbbbba054c2d16d4" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 33367 } } peers { permanent_uuid: "29e27b15af2b46f2882033ac1b6f5e79" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39827 } }
I20260809 15:10:24.670406 14392 raft_consensus.cc:374] T 00000000000000000000000000000000 P af55f2df9c734fa1bbbbba054c2d16d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:10:24.670531 14392 raft_consensus.cc:729] T 00000000000000000000000000000000 P af55f2df9c734fa1bbbbba054c2d16d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af55f2df9c734fa1bbbbba054c2d16d4, State: Initialized, Role: FOLLOWER
I20260809 15:10:24.671093 14392 consensus_queue.cc:260] T 00000000000000000000000000000000 P af55f2df9c734fa1bbbbba054c2d16d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332a2583ad1749b0ba5a740501c90c7f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 37503 } } peers { permanent_uuid: "af55f2df9c734fa1bbbbba054c2d16d4" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 33367 } } peers { permanent_uuid: "29e27b15af2b46f2882033ac1b6f5e79" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39827 } }
I20260809 15:10:24.672003 14582 sys_catalog.cc:455] T 00000000000000000000000000000000 P af55f2df9c734fa1bbbbba054c2d16d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332a2583ad1749b0ba5a740501c90c7f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 37503 } } peers { permanent_uuid: "af55f2df9c734fa1bbbbba054c2d16d4" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 33367 } } peers { permanent_uuid: "29e27b15af2b46f2882033ac1b6f5e79" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39827 } } }
I20260809 15:10:24.672427 14582 sys_catalog.cc:458] T 00000000000000000000000000000000 P af55f2df9c734fa1bbbbba054c2d16d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:10:24.673170 14392 sys_catalog.cc:564] T 00000000000000000000000000000000 P af55f2df9c734fa1bbbbba054c2d16d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:10:24.692727 14278 master_runner.cc:263] No leader master found from master af55f2df9c734fa1bbbbba054c2d16d4
I20260809 15:10:24.692876 14278 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 15:10:24.697724 14607 catalog_manager.cc:1558] T 00000000000000000000000000000000 P af55f2df9c734fa1bbbbba054c2d16d4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:10:24.697842 14607 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:10:24.748051 14582 raft_consensus.cc:482] T 00000000000000000000000000000000 P af55f2df9c734fa1bbbbba054c2d16d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:10:24.748435 14582 raft_consensus.cc:504] T 00000000000000000000000000000000 P af55f2df9c734fa1bbbbba054c2d16d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332a2583ad1749b0ba5a740501c90c7f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 37503 } } peers { permanent_uuid: "af55f2df9c734fa1bbbbba054c2d16d4" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 33367 } } peers { permanent_uuid: "29e27b15af2b46f2882033ac1b6f5e79" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39827 } }
I20260809 15:10:24.750202 14582 leader_election.cc:288] T 00000000000000000000000000000000 P af55f2df9c734fa1bbbbba054c2d16d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 332a2583ad1749b0ba5a740501c90c7f (127.24.183.254:37503), 29e27b15af2b46f2882033ac1b6f5e79 (127.24.183.252:39827)
I20260809 15:10:24.751340 14476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "af55f2df9c734fa1bbbbba054c2d16d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29e27b15af2b46f2882033ac1b6f5e79" is_pre_election: true
I20260809 15:10:24.752048 14476 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 29e27b15af2b46f2882033ac1b6f5e79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af55f2df9c734fa1bbbbba054c2d16d4 in term 0.
I20260809 15:10:24.752966 14229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "af55f2df9c734fa1bbbbba054c2d16d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "332a2583ad1749b0ba5a740501c90c7f" is_pre_election: true
I20260809 15:10:24.753558 14229 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 332a2583ad1749b0ba5a740501c90c7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af55f2df9c734fa1bbbbba054c2d16d4 in term 0.
I20260809 15:10:24.754190 14316 leader_election.cc:302] T 00000000000000000000000000000000 P af55f2df9c734fa1bbbbba054c2d16d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29e27b15af2b46f2882033ac1b6f5e79, af55f2df9c734fa1bbbbba054c2d16d4; no voters:
I20260809 15:10:24.755110 14582 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af55f2df9c734fa1bbbbba054c2d16d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:10:24.755273 14582 raft_consensus.cc:482] T 00000000000000000000000000000000 P af55f2df9c734fa1bbbbba054c2d16d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:10:24.755385 14582 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af55f2df9c734fa1bbbbba054c2d16d4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:10:24.758944 14582 raft_consensus.cc:504] T 00000000000000000000000000000000 P af55f2df9c734fa1bbbbba054c2d16d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332a2583ad1749b0ba5a740501c90c7f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 37503 } } peers { permanent_uuid: "af55f2df9c734fa1bbbbba054c2d16d4" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 33367 } } peers { permanent_uuid: "29e27b15af2b46f2882033ac1b6f5e79" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39827 } }
I20260809 15:10:24.760025 14229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "af55f2df9c734fa1bbbbba054c2d16d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "332a2583ad1749b0ba5a740501c90c7f"
I20260809 15:10:24.760248 14229 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 332a2583ad1749b0ba5a740501c90c7f [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:10:24.763664 14229 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 332a2583ad1749b0ba5a740501c90c7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af55f2df9c734fa1bbbbba054c2d16d4 in term 1.
I20260809 15:10:24.769577 14315 leader_election.cc:302] T 00000000000000000000000000000000 P af55f2df9c734fa1bbbbba054c2d16d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 332a2583ad1749b0ba5a740501c90c7f, af55f2df9c734fa1bbbbba054c2d16d4; no voters:
I20260809 15:10:24.773694 14476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "af55f2df9c734fa1bbbbba054c2d16d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29e27b15af2b46f2882033ac1b6f5e79"
I20260809 15:10:24.773962 14476 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29e27b15af2b46f2882033ac1b6f5e79 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:10:24.778085 14476 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 29e27b15af2b46f2882033ac1b6f5e79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af55f2df9c734fa1bbbbba054c2d16d4 in term 1.
I20260809 15:10:24.778699 14582 leader_election.cc:288] T 00000000000000000000000000000000 P af55f2df9c734fa1bbbbba054c2d16d4 [CANDIDATE]: Term 1 election: Requested vote from peers 332a2583ad1749b0ba5a740501c90c7f (127.24.183.254:37503), 29e27b15af2b46f2882033ac1b6f5e79 (127.24.183.252:39827)
I20260809 15:10:24.779253 14582 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af55f2df9c734fa1bbbbba054c2d16d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:10:24.779425 14582 raft_consensus.cc:686] T 00000000000000000000000000000000 P af55f2df9c734fa1bbbbba054c2d16d4 [term 1 LEADER]: Becoming Leader. State: Replica: af55f2df9c734fa1bbbbba054c2d16d4, State: Running, Role: LEADER
I20260809 15:10:24.780543 14582 consensus_queue.cc:237] T 00000000000000000000000000000000 P af55f2df9c734fa1bbbbba054c2d16d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332a2583ad1749b0ba5a740501c90c7f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 37503 } } peers { permanent_uuid: "af55f2df9c734fa1bbbbba054c2d16d4" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 33367 } } peers { permanent_uuid: "29e27b15af2b46f2882033ac1b6f5e79" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39827 } }
I20260809 15:10:24.798425 14609 sys_catalog.cc:455] T 00000000000000000000000000000000 P af55f2df9c734fa1bbbbba054c2d16d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af55f2df9c734fa1bbbbba054c2d16d4. Latest consensus state: current_term: 1 leader_uuid: "af55f2df9c734fa1bbbbba054c2d16d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332a2583ad1749b0ba5a740501c90c7f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 37503 } } peers { permanent_uuid: "af55f2df9c734fa1bbbbba054c2d16d4" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 33367 } } peers { permanent_uuid: "29e27b15af2b46f2882033ac1b6f5e79" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39827 } } }
I20260809 15:10:24.798867 14609 sys_catalog.cc:458] T 00000000000000000000000000000000 P af55f2df9c734fa1bbbbba054c2d16d4 [sys.catalog]: This master's current role is: LEADER
I20260809 15:10:24.805796 14613 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:10:24.808275 14613 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:10:24.832167 14476 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 29e27b15af2b46f2882033ac1b6f5e79 [term 1 FOLLOWER]: Refusing update from remote peer af55f2df9c734fa1bbbbba054c2d16d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:10:24.833371 14609 consensus_queue.cc:1035] T 00000000000000000000000000000000 P af55f2df9c734fa1bbbbba054c2d16d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29e27b15af2b46f2882033ac1b6f5e79" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:10:24.842301 14229 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 332a2583ad1749b0ba5a740501c90c7f [term 1 FOLLOWER]: Refusing update from remote peer af55f2df9c734fa1bbbbba054c2d16d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:10:24.843297 14609 consensus_queue.cc:1035] T 00000000000000000000000000000000 P af55f2df9c734fa1bbbbba054c2d16d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "332a2583ad1749b0ba5a740501c90c7f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 37503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:10:24.867446 14527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29e27b15af2b46f2882033ac1b6f5e79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af55f2df9c734fa1bbbbba054c2d16d4. Latest consensus state: current_term: 1 leader_uuid: "af55f2df9c734fa1bbbbba054c2d16d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332a2583ad1749b0ba5a740501c90c7f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 37503 } } peers { permanent_uuid: "af55f2df9c734fa1bbbbba054c2d16d4" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 33367 } } peers { permanent_uuid: "29e27b15af2b46f2882033ac1b6f5e79" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39827 } } }
I20260809 15:10:24.867820 14527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29e27b15af2b46f2882033ac1b6f5e79 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:10:24.872200 14616 mvcc.cc:205] Tried to move back new op lower bound from 7316636568899514368 to 7316636568714944512. Current Snapshot: MvccSnapshot[applied={T|T < 7316636568899514368}]
I20260809 15:10:24.878865 14615 mvcc.cc:205] Tried to move back new op lower bound from 7316636568899514368 to 7316636568714944512. Current Snapshot: MvccSnapshot[applied={T|T < 7316636568899514368}]
I20260809 15:10:24.879806 14582 sys_catalog.cc:455] T 00000000000000000000000000000000 P af55f2df9c734fa1bbbbba054c2d16d4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af55f2df9c734fa1bbbbba054c2d16d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332a2583ad1749b0ba5a740501c90c7f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 37503 } } peers { permanent_uuid: "af55f2df9c734fa1bbbbba054c2d16d4" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 33367 } } peers { permanent_uuid: "29e27b15af2b46f2882033ac1b6f5e79" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39827 } } }
I20260809 15:10:24.880142 14582 sys_catalog.cc:458] T 00000000000000000000000000000000 P af55f2df9c734fa1bbbbba054c2d16d4 [sys.catalog]: This master's current role is: LEADER
I20260809 15:10:24.879994 14527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29e27b15af2b46f2882033ac1b6f5e79 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af55f2df9c734fa1bbbbba054c2d16d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332a2583ad1749b0ba5a740501c90c7f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 37503 } } peers { permanent_uuid: "af55f2df9c734fa1bbbbba054c2d16d4" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 33367 } } peers { permanent_uuid: "29e27b15af2b46f2882033ac1b6f5e79" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39827 } } }
I20260809 15:10:24.880271 14527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29e27b15af2b46f2882033ac1b6f5e79 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:10:24.882139 14613 catalog_manager.cc:1347] Generated new cluster ID: 7d9b720c6f3b4f6a87f9e6571d1136a2
I20260809 15:10:24.882241 14613 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:10:24.899044 14553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 332a2583ad1749b0ba5a740501c90c7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader af55f2df9c734fa1bbbbba054c2d16d4. Latest consensus state: current_term: 1 leader_uuid: "af55f2df9c734fa1bbbbba054c2d16d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332a2583ad1749b0ba5a740501c90c7f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 37503 } } peers { permanent_uuid: "af55f2df9c734fa1bbbbba054c2d16d4" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 33367 } } peers { permanent_uuid: "29e27b15af2b46f2882033ac1b6f5e79" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39827 } } }
I20260809 15:10:24.899389 14553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 332a2583ad1749b0ba5a740501c90c7f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:10:24.902156 14582 sys_catalog.cc:455] T 00000000000000000000000000000000 P af55f2df9c734fa1bbbbba054c2d16d4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af55f2df9c734fa1bbbbba054c2d16d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332a2583ad1749b0ba5a740501c90c7f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 37503 } } peers { permanent_uuid: "af55f2df9c734fa1bbbbba054c2d16d4" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 33367 } } peers { permanent_uuid: "29e27b15af2b46f2882033ac1b6f5e79" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39827 } } }
I20260809 15:10:24.902521 14582 sys_catalog.cc:458] T 00000000000000000000000000000000 P af55f2df9c734fa1bbbbba054c2d16d4 [sys.catalog]: This master's current role is: LEADER
I20260809 15:10:24.918640 14553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 332a2583ad1749b0ba5a740501c90c7f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af55f2df9c734fa1bbbbba054c2d16d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332a2583ad1749b0ba5a740501c90c7f" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 37503 } } peers { permanent_uuid: "af55f2df9c734fa1bbbbba054c2d16d4" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 33367 } } peers { permanent_uuid: "29e27b15af2b46f2882033ac1b6f5e79" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 39827 } } }
I20260809 15:10:24.918905 14553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 332a2583ad1749b0ba5a740501c90c7f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:10:24.937179 14613 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:10:24.939495 14613 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:10:24.955163 14613 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af55f2df9c734fa1bbbbba054c2d16d4: Generated new TSK 0
I20260809 15:10:24.956331 14613 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:10:25.536742 14548 catalog_manager.cc:1259] Loaded cluster ID: 7d9b720c6f3b4f6a87f9e6571d1136a2
I20260809 15:10:25.536860 14548 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 29e27b15af2b46f2882033ac1b6f5e79: loading cluster ID for follower catalog manager: success
I20260809 15:10:25.539465 14548 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 29e27b15af2b46f2882033ac1b6f5e79: acquiring CA information for follower catalog manager: success
I20260809 15:10:25.541365 14548 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 29e27b15af2b46f2882033ac1b6f5e79: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 15:10:25.591536 14574 catalog_manager.cc:1259] Loaded cluster ID: 7d9b720c6f3b4f6a87f9e6571d1136a2
I20260809 15:10:25.591645 14574 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 332a2583ad1749b0ba5a740501c90c7f: loading cluster ID for follower catalog manager: success
I20260809 15:10:25.594321 14574 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 332a2583ad1749b0ba5a740501c90c7f: acquiring CA information for follower catalog manager: success
I20260809 15:10:25.596473 14574 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 332a2583ad1749b0ba5a740501c90c7f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 15:10:26.112574 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 14161
I20260809 15:10:26.113952 14261 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:10:26.225526 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 14161
I20260809 15:10:26.252820 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 14278
I20260809 15:10:26.254007 14386 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:10:26.362887 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 14278
I20260809 15:10:26.388159 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 14401
I20260809 15:10:26.389238 14503 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:10:26.498970 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 14401
2026-08-09T15:10:26Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5007 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:10:26 dist-test-slave-nj21 krb5kdc[14641](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:10:26 dist-test-slave-nj21 krb5kdc[14641](info): set up 2 sockets
Aug 09 15:10:26 dist-test-slave-nj21 krb5kdc[14641](info): commencing operation
krb5kdc: starting...
W20260809 15:10:27.719034 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.000s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:10:27 dist-test-slave-nj21 krb5kdc[14641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288227, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:10:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:10:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
I20260809 15:10:27.981338 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:36289
--webserver_interface=127.24.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:40073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:36289
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:10:28.316435 14659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:10:28.316743 14659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:10:28.316879 14659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:10:28.324070 14659 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:10:28.324184 14659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:10:28.324272 14659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:10:28.324347 14659 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:10:28.324409 14659 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:10:28.334149 14659 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:40073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:36289
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:36289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:10:28.335445 14659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:10:28.336743 14659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:10:28.348136 14673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:28.348268 14670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:28.348521 14669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:10:28.349223 14659 server_base.cc:1034] running on GCE node
I20260809 15:10:28.349963 14659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:10:28.351245 14659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:10:28.352433 14659 hybrid_clock.cc:648] HybridClock initialized: now 1786288228352380 us; error 38 us; skew 500 ppm
I20260809 15:10:28.352763 14659 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:10:28 dist-test-slave-nj21 krb5kdc[14641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288228, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:10:28.355782 14659 init.cc:377] Logged in from keytab as kudu/127.24.183.254@KRBTEST.COM (short username kudu)
I20260809 15:10:28.357142 14659 webserver.cc:460] Webserver started at http://127.24.183.254:46677/ using document root <none> and password file <none>
I20260809 15:10:28.357752 14659 fs_manager.cc:359] Metadata directory not provided
I20260809 15:10:28.357851 14659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:10:28.358106 14659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:10:28.361032 14659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "1822c70afbc64125b940a683ef28efc9"
format_stamp: "Formatted at 2026-08-09 15:10:28 on dist-test-slave-nj21"
server_key: "3e2c285e73cc9138afb075eb779c062e"
server_key_iv: "443b8dc4a2794085d6305cfd8a3d4fb0"
server_key_version: "encryptionkey@0"
I20260809 15:10:28.361714 14659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "1822c70afbc64125b940a683ef28efc9"
format_stamp: "Formatted at 2026-08-09 15:10:28 on dist-test-slave-nj21"
server_key: "3e2c285e73cc9138afb075eb779c062e"
server_key_iv: "443b8dc4a2794085d6305cfd8a3d4fb0"
server_key_version: "encryptionkey@0"
I20260809 15:10:28.366668 14659 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 15:10:28.369982 14686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:28.371225 14659 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:10:28.371392 14659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "1822c70afbc64125b940a683ef28efc9"
format_stamp: "Formatted at 2026-08-09 15:10:28 on dist-test-slave-nj21"
server_key: "3e2c285e73cc9138afb075eb779c062e"
server_key_iv: "443b8dc4a2794085d6305cfd8a3d4fb0"
server_key_version: "encryptionkey@0"
I20260809 15:10:28.371567 14659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:10:28.423596 14659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:10:28.428042 14659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:10:28.428321 14659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:10:28.446575 14659 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:36289
I20260809 15:10:28.446596 14774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:36289 every 8 connection(s)
I20260809 15:10:28.447871 14659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:10:28.451545 14776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:10:28.455690 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 14659
I20260809 15:10:28.455896 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:10:28.456539 25311 external_mini_cluster.cc:1428] Setting key 1406027459e6bb12859a5fc15db62c04
I20260809 15:10:28.463344 14776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1822c70afbc64125b940a683ef28efc9: Bootstrap starting.
I20260809 15:10:28.466892 14776 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1822c70afbc64125b940a683ef28efc9: Neither blocks nor log segments found. Creating new log.
I20260809 15:10:28.468036 14776 log.cc:824] T 00000000000000000000000000000000 P 1822c70afbc64125b940a683ef28efc9: Log is configured to *not* fsync() on all Append() calls
I20260809 15:10:28.470855 14776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1822c70afbc64125b940a683ef28efc9: No bootstrap required, opened a new log
Aug 09 15:10:28 dist-test-slave-nj21 krb5kdc[14641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288227, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:10:28.480017 14776 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1822c70afbc64125b940a683ef28efc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1822c70afbc64125b940a683ef28efc9" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36289 } }
I20260809 15:10:28.480319 14780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:28.458254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55384 (local address 127.24.183.254:36289)
0809 15:10:28.459095 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:28.459111 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:28.459169 (+ 58us) server_negotiation.cc:407] Connection header received
0809 15:10:28.461682 (+ 2513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:28.461710 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:28.462171 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:28.462599 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:28.464033 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:28.466034 (+ 2001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:28.467423 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:28.468451 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:28.468644 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:28.472976 (+ 4332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:28.472987 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:28.473001 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:28.473050 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:28.476247 (+ 3197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:28.477076 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:28.477082 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:28.477088 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:28.477253 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:28.477638 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:28.477641 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:28.477643 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:28.478157 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:28.478370 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:28.478900 (+ 530us) server_negotiation.cc:299] Negotiation successful
0809 15:10:28.479282 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":353,"threads_started":1}
I20260809 15:10:28.480365 14776 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1822c70afbc64125b940a683ef28efc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:10:28.480479 14776 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1822c70afbc64125b940a683ef28efc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1822c70afbc64125b940a683ef28efc9, State: Initialized, Role: FOLLOWER
I20260809 15:10:28.481172 14776 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1822c70afbc64125b940a683ef28efc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1822c70afbc64125b940a683ef28efc9" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36289 } }
I20260809 15:10:28.481474 14776 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1822c70afbc64125b940a683ef28efc9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:10:28.481630 14776 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1822c70afbc64125b940a683ef28efc9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:10:28.481814 14776 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1822c70afbc64125b940a683ef28efc9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:10:28.484305 25311 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:10:28.484462 14776 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1822c70afbc64125b940a683ef28efc9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1822c70afbc64125b940a683ef28efc9" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36289 } }
I20260809 15:10:28.484920 14776 leader_election.cc:302] T 00000000000000000000000000000000 P 1822c70afbc64125b940a683ef28efc9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1822c70afbc64125b940a683ef28efc9; no voters:
I20260809 15:10:28.485472 14776 leader_election.cc:288] T 00000000000000000000000000000000 P 1822c70afbc64125b940a683ef28efc9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:10:28.485608 14783 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1822c70afbc64125b940a683ef28efc9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:10:28.487154 14783 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1822c70afbc64125b940a683ef28efc9 [term 1 LEADER]: Becoming Leader. State: Replica: 1822c70afbc64125b940a683ef28efc9, State: Running, Role: LEADER
I20260809 15:10:28.487473 14776 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1822c70afbc64125b940a683ef28efc9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:10:28.487780 14783 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1822c70afbc64125b940a683ef28efc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1822c70afbc64125b940a683ef28efc9" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36289 } }
I20260809 15:10:28.491062 14785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1822c70afbc64125b940a683ef28efc9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1822c70afbc64125b940a683ef28efc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1822c70afbc64125b940a683ef28efc9" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36289 } } }
I20260809 15:10:28.491096 14786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1822c70afbc64125b940a683ef28efc9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1822c70afbc64125b940a683ef28efc9. Latest consensus state: current_term: 1 leader_uuid: "1822c70afbc64125b940a683ef28efc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1822c70afbc64125b940a683ef28efc9" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36289 } } }
I20260809 15:10:28.491557 14785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1822c70afbc64125b940a683ef28efc9 [sys.catalog]: This master's current role is: LEADER
I20260809 15:10:28.491555 14786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1822c70afbc64125b940a683ef28efc9 [sys.catalog]: This master's current role is: LEADER
I20260809 15:10:28.493067 14796 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:10:28.498361 14796 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:10:28.506721 14796 catalog_manager.cc:1347] Generated new cluster ID: 9a183ebe8c42420e892f9f02dcec89e1
I20260809 15:10:28.506855 14796 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:10:28.517270 14796 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:10:28.518476 14796 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:10:28.532078 14796 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1822c70afbc64125b940a683ef28efc9: Generated new TSK 0
I20260809 15:10:28.532795 14796 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:10:28.545658 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 14659
I20260809 15:10:28.546758 14766 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:10:28.638032 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 14659
2026-08-09T15:10:28Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2135 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:10:28 dist-test-slave-nj21 krb5kdc[14825](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:10:28 dist-test-slave-nj21 krb5kdc[14825](info): set up 2 sockets
Aug 09 15:10:28 dist-test-slave-nj21 krb5kdc[14825](info): commencing operation
krb5kdc: starting...
W20260809 15:10:29.858829 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.003s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:10:30 dist-test-slave-nj21 krb5kdc[14825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:10:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:10:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
I20260809 15:10:30.112498 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:41811
--webserver_interface=127.24.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:33207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:41811
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:10:30.442634 14845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:10:30.442910 14845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:10:30.443009 14845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:10:30.450045 14845 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:10:30.450139 14845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:10:30.450199 14845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:10:30.450256 14845 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:10:30.450300 14845 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:10:30.459628 14845 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:33207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:41811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:41811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:10:30.460875 14845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:10:30.462138 14845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:10:30.473237 14854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:30.473197 14852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:30.473268 14856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:10:30.473968 14845 server_base.cc:1034] running on GCE node
I20260809 15:10:30.474690 14845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:10:30.476073 14845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:10:30.477275 14845 hybrid_clock.cc:648] HybridClock initialized: now 1786288230477223 us; error 35 us; skew 500 ppm
I20260809 15:10:30.477586 14845 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:10:30 dist-test-slave-nj21 krb5kdc[14825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288230, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:10:30.485877 14845 init.cc:377] Logged in from keytab as kudu/127.24.183.254@KRBTEST.COM (short username kudu)
I20260809 15:10:30.487356 14845 webserver.cc:460] Webserver started at http://127.24.183.254:41829/ using document root <none> and password file <none>
I20260809 15:10:30.488112 14845 fs_manager.cc:359] Metadata directory not provided
I20260809 15:10:30.488234 14845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:10:30.488523 14845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:10:30.491484 14845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "49997ba886fa4ba9b145e19999ab8951"
format_stamp: "Formatted at 2026-08-09 15:10:30 on dist-test-slave-nj21"
server_key: "90039f93bb59fa1a23ff0e4d8ba72425"
server_key_iv: "b1505f0f0c2c6d93cd9902ca659cdeb1"
server_key_version: "encryptionkey@0"
I20260809 15:10:30.492264 14845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "49997ba886fa4ba9b145e19999ab8951"
format_stamp: "Formatted at 2026-08-09 15:10:30 on dist-test-slave-nj21"
server_key: "90039f93bb59fa1a23ff0e4d8ba72425"
server_key_iv: "b1505f0f0c2c6d93cd9902ca659cdeb1"
server_key_version: "encryptionkey@0"
I20260809 15:10:30.497206 14845 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 15:10:30.500468 14866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:30.501715 14845 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:10:30.501878 14845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "49997ba886fa4ba9b145e19999ab8951"
format_stamp: "Formatted at 2026-08-09 15:10:30 on dist-test-slave-nj21"
server_key: "90039f93bb59fa1a23ff0e4d8ba72425"
server_key_iv: "b1505f0f0c2c6d93cd9902ca659cdeb1"
server_key_version: "encryptionkey@0"
I20260809 15:10:30.502051 14845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:10:30.517313 14845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:10:30.521375 14845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:10:30.521636 14845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:10:30.539342 14845 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:41811
I20260809 15:10:30.539362 14963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:41811 every 8 connection(s)
I20260809 15:10:30.540668 14845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:10:30.544333 14964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:10:30.544977 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 14845
I20260809 15:10:30.545209 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:10:30.546092 25311 external_mini_cluster.cc:1428] Setting key ba29b5b99173d03009d52467a18d0e0f
I20260809 15:10:30.558430 14964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951: Bootstrap starting.
Aug 09 15:10:30 dist-test-slave-nj21 krb5kdc[14825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:10:30.561952 14964 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951: Neither blocks nor log segments found. Creating new log.
I20260809 15:10:30.563257 14964 log.cc:824] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951: Log is configured to *not* fsync() on all Append() calls
I20260809 15:10:30.566409 14964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951: No bootstrap required, opened a new log
I20260809 15:10:30.570026 14974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:30.548242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56522 (local address 127.24.183.254:41811)
0809 15:10:30.549009 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:30.549021 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:30.549067 (+ 46us) server_negotiation.cc:407] Connection header received
0809 15:10:30.550732 (+ 1665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:30.550753 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:30.551109 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:30.551425 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:30.553152 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:30.554468 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:30.556301 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:30.556944 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:30.557075 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:30.561872 (+ 4797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:30.561884 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:30.561902 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:30.561957 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:30.565503 (+ 3546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:30.566175 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:30.566183 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:30.566190 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:30.566409 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:30.566850 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:30.566858 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:30.566861 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:30.567529 (+ 668us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:30.567894 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:30.568235 (+ 341us) server_negotiation.cc:299] Negotiation successful
0809 15:10:30.568680 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":563,"thread_start_us":354,"threads_started":1}
I20260809 15:10:30.575604 14964 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49997ba886fa4ba9b145e19999ab8951" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 41811 } }
I20260809 15:10:30.575989 14964 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:10:30.576064 14964 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49997ba886fa4ba9b145e19999ab8951, State: Initialized, Role: FOLLOWER
I20260809 15:10:30.576534 14964 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49997ba886fa4ba9b145e19999ab8951" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 41811 } }
I20260809 15:10:30.576728 14964 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:10:30.576821 14964 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:10:30.576943 14964 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:10:30.578959 14964 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49997ba886fa4ba9b145e19999ab8951" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 41811 } }
I20260809 15:10:30.579383 14964 leader_election.cc:302] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49997ba886fa4ba9b145e19999ab8951; no voters:
I20260809 15:10:30.579909 14964 leader_election.cc:288] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:10:30.580022 14976 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:10:30.580700 14976 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [term 1 LEADER]: Becoming Leader. State: Replica: 49997ba886fa4ba9b145e19999ab8951, State: Running, Role: LEADER
I20260809 15:10:30.581287 14976 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49997ba886fa4ba9b145e19999ab8951" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 41811 } }
I20260809 15:10:30.581871 14964 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:10:30.584223 14979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49997ba886fa4ba9b145e19999ab8951. Latest consensus state: current_term: 1 leader_uuid: "49997ba886fa4ba9b145e19999ab8951" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49997ba886fa4ba9b145e19999ab8951" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 41811 } } }
I20260809 15:10:30.584196 14978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49997ba886fa4ba9b145e19999ab8951" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49997ba886fa4ba9b145e19999ab8951" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 41811 } } }
I20260809 15:10:30.584657 14978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [sys.catalog]: This master's current role is: LEADER
I20260809 15:10:30.584657 14979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [sys.catalog]: This master's current role is: LEADER
I20260809 15:10:30.586449 14987 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:10:30.591241 14987 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:10:30.600188 14987 catalog_manager.cc:1347] Generated new cluster ID: 67744b2672344797b8ec69afdbdf09a1
I20260809 15:10:30.600281 14987 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:10:30.619161 14987 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:10:30.620472 14987 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:10:30.636966 14987 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951: Generated new TSK 0
I20260809 15:10:30.637843 14987 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab.
Entry for principal kudu/127.24.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab.
I20260809 15:10:30.715234 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.193:0
--local_ip_for_outbound_sockets=127.24.183.193
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:41811
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:33207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:10:31.081890 15017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:10:31.082172 15017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:10:31.082274 15017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:10:31.089805 15017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:10:31.089910 15017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:10:31.090034 15017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.193
I20260809 15:10:31.099572 15017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:33207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:41811
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.193
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:10:31.100893 15017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:10:31.102246 15017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:10:31.115527 15027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:31.115556 15032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:31.115604 15028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:10:31.116074 15017 server_base.cc:1034] running on GCE node
I20260809 15:10:31.116755 15017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:10:31.117647 15017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:10:31.118846 15017 hybrid_clock.cc:648] HybridClock initialized: now 1786288231118788 us; error 48 us; skew 500 ppm
I20260809 15:10:31.119133 15017 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:10:31 dist-test-slave-nj21 krb5kdc[14825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288231, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:10:31.122174 15017 init.cc:377] Logged in from keytab as kudu/127.24.183.193@KRBTEST.COM (short username kudu)
I20260809 15:10:31.123530 15017 webserver.cc:460] Webserver started at http://127.24.183.193:35243/ using document root <none> and password file <none>
I20260809 15:10:31.124209 15017 fs_manager.cc:359] Metadata directory not provided
I20260809 15:10:31.124346 15017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:10:31.124635 15017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:10:31.127532 15017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/data/instance:
uuid: "4e57e56718f64d37a9ee564ebad08a4e"
format_stamp: "Formatted at 2026-08-09 15:10:31 on dist-test-slave-nj21"
server_key: "cf13abac65bace41da3d311e65d63473"
server_key_iv: "74bb9cf8d79954e751ec3d9cb04aab06"
server_key_version: "encryptionkey@0"
I20260809 15:10:31.128304 15017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance:
uuid: "4e57e56718f64d37a9ee564ebad08a4e"
format_stamp: "Formatted at 2026-08-09 15:10:31 on dist-test-slave-nj21"
server_key: "cf13abac65bace41da3d311e65d63473"
server_key_iv: "74bb9cf8d79954e751ec3d9cb04aab06"
server_key_version: "encryptionkey@0"
I20260809 15:10:31.133383 15017 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 15:10:31.137032 15039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:31.138335 15017 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 15:10:31.138494 15017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/wal
uuid: "4e57e56718f64d37a9ee564ebad08a4e"
format_stamp: "Formatted at 2026-08-09 15:10:31 on dist-test-slave-nj21"
server_key: "cf13abac65bace41da3d311e65d63473"
server_key_iv: "74bb9cf8d79954e751ec3d9cb04aab06"
server_key_version: "encryptionkey@0"
I20260809 15:10:31.138664 15017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:10:31.160207 15017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:10:31.163925 15017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:10:31.164194 15017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:10:31.165186 15017 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:10:31.166512 15017 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:10:31.166599 15017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:31.166718 15017 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:10:31.166771 15017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:31.197558 15017 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.193:37317
I20260809 15:10:31.197600 15252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.193:37317 every 8 connection(s)
I20260809 15:10:31.198848 15017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
I20260809 15:10:31.204460 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 15017
I20260809 15:10:31.204679 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance
I20260809 15:10:31.205389 25311 external_mini_cluster.cc:1428] Setting key e53981864f90e46bf0171b344ffc1e59
Aug 09 15:10:31 dist-test-slave-nj21 krb5kdc[14825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288231, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:10:31.218606 15255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:31.201349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:36461 (local address 127.24.183.254:41811)
0809 15:10:31.201816 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:31.201828 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:31.202649 (+ 821us) server_negotiation.cc:407] Connection header received
0809 15:10:31.204525 (+ 1876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:31.204530 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:31.204617 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:31.204721 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:31.206349 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:31.207352 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:31.208889 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:31.209292 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:31.209406 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:31.213094 (+ 3688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:31.213099 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:31.213103 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:31.213136 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:31.215615 (+ 2479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:31.216435 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:31.216438 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:31.216441 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:31.216566 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:31.217005 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:31.217008 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:31.217010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:31.217271 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:31.217379 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:31.218129 (+ 750us) server_negotiation.cc:299] Negotiation successful
0809 15:10:31.218307 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":245,"threads_started":1}
I20260809 15:10:31.219655 15256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:31.201787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.193:36461)
0809 15:10:31.202506 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:31.202543 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:31.204259 (+ 1716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:31.204968 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:31.204976 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:31.205558 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:10:31.206167 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:31.206185 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:31.207496 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:31.207500 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:31.208741 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:31.208753 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:31.209532 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:31.209539 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:31.209766 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:31.210398 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:10:31.210423 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:10:31.212886 (+ 2463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:10:31.215789 (+ 2903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:31.215797 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:31.215812 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:31.216335 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:31.216656 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:31.216661 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:31.216664 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:31.216909 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:31.217384 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:10:31.217391 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:10:31.217768 (+ 377us) client_negotiation.cc:769] Sending connection context
0809 15:10:31.218051 (+ 283us) client_negotiation.cc:240] Negotiation successful
0809 15:10:31.218559 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":262,"threads_started":1}
I20260809 15:10:31.221140 15253 heartbeater.cc:347] Connected to a master server at 127.24.183.254:41811
I20260809 15:10:31.221415 15253 heartbeater.cc:464] Registering TS with master...
I20260809 15:10:31.222128 15253 heartbeater.cc:511] Master 127.24.183.254:41811 requested a full tablet report, sending...
I20260809 15:10:31.224195 14900 ts_manager.cc:194] Registered new tserver with Master: 4e57e56718f64d37a9ee564ebad08a4e (127.24.183.193:37317)
I20260809 15:10:31.225610 14900 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.193@KRBTEST.COM'} at 127.24.183.193:36461
WARNING: no policy specified for kudu/127.24.183.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab.
Entry for principal kudu/127.24.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab.
I20260809 15:10:31.262769 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.194:0
--local_ip_for_outbound_sockets=127.24.183.194
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:41811
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:33207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:10:31.599761 15260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:10:31.600039 15260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:10:31.600178 15260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:10:31.607381 15260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:10:31.607492 15260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:10:31.607640 15260 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.194
I20260809 15:10:31.617118 15260 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:33207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:41811
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.194
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:10:31.618391 15260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:10:31.619730 15260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:10:31.632298 15275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:31.632225 15279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:31.632313 15277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:10:31.633134 15260 server_base.cc:1034] running on GCE node
I20260809 15:10:31.633703 15260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:10:31.634585 15260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:10:31.635756 15260 hybrid_clock.cc:648] HybridClock initialized: now 1786288231635703 us; error 39 us; skew 500 ppm
I20260809 15:10:31.636029 15260 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:10:31 dist-test-slave-nj21 krb5kdc[14825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288231, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:10:31.639067 15260 init.cc:377] Logged in from keytab as kudu/127.24.183.194@KRBTEST.COM (short username kudu)
I20260809 15:10:31.640487 15260 webserver.cc:460] Webserver started at http://127.24.183.194:33949/ using document root <none> and password file <none>
I20260809 15:10:31.641139 15260 fs_manager.cc:359] Metadata directory not provided
I20260809 15:10:31.641255 15260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:10:31.641556 15260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:10:31.644580 15260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/data/instance:
uuid: "0b26588483d746bdba86f18f39b0ca50"
format_stamp: "Formatted at 2026-08-09 15:10:31 on dist-test-slave-nj21"
server_key: "0daae33d260d9132d5fb962faa6f6c75"
server_key_iv: "98aa459986d440a9d2a7d13bfa7fc021"
server_key_version: "encryptionkey@0"
I20260809 15:10:31.645323 15260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance:
uuid: "0b26588483d746bdba86f18f39b0ca50"
format_stamp: "Formatted at 2026-08-09 15:10:31 on dist-test-slave-nj21"
server_key: "0daae33d260d9132d5fb962faa6f6c75"
server_key_iv: "98aa459986d440a9d2a7d13bfa7fc021"
server_key_version: "encryptionkey@0"
I20260809 15:10:31.650439 15260 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 15:10:31.653751 15289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:31.655038 15260 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:10:31.655190 15260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/wal
uuid: "0b26588483d746bdba86f18f39b0ca50"
format_stamp: "Formatted at 2026-08-09 15:10:31 on dist-test-slave-nj21"
server_key: "0daae33d260d9132d5fb962faa6f6c75"
server_key_iv: "98aa459986d440a9d2a7d13bfa7fc021"
server_key_version: "encryptionkey@0"
I20260809 15:10:31.655361 15260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:10:31.669880 15260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:10:31.673653 15260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:10:31.673913 15260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:10:31.674937 15260 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:10:31.678586 15260 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:10:31.678678 15260 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:31.678802 15260 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:10:31.678871 15260 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:31.711782 15260 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.194:46385
I20260809 15:10:31.711853 15482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.194:46385 every 8 connection(s)
I20260809 15:10:31.713050 15260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
I20260809 15:10:31.715931 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 15260
I20260809 15:10:31.716138 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance
I20260809 15:10:31.716571 25311 external_mini_cluster.cc:1428] Setting key 2780c9170c27bb18ffd1bc058045465f
Aug 09 15:10:31 dist-test-slave-nj21 krb5kdc[14825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288231, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:10:31.734401 15255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:31.715756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:48347 (local address 127.24.183.254:41811)
0809 15:10:31.715981 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:31.715988 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:31.717276 (+ 1288us) server_negotiation.cc:407] Connection header received
0809 15:10:31.719546 (+ 2270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:31.719551 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:31.719629 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:31.719745 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:31.721376 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:31.722289 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:31.723968 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:31.724557 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:31.724673 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:31.728279 (+ 3606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:31.728285 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:31.728290 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:31.728329 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:31.731083 (+ 2754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:31.731879 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:31.731886 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:31.731890 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:31.732071 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:31.732541 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:31.732547 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:31.732551 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:31.732861 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:31.733012 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:31.733737 (+ 725us) server_negotiation.cc:299] Negotiation successful
0809 15:10:31.733988 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 15:10:31.735380 15486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:31.716233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.194:48347)
0809 15:10:31.717107 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:31.717155 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:31.719286 (+ 2131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:31.719977 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:31.719984 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:31.720611 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:10:31.721207 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:31.721226 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:31.722459 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:31.722464 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:31.723812 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:31.723826 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:31.724822 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:31.724828 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:31.725049 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:31.725765 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:10:31.725790 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:10:31.728057 (+ 2267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:10:31.731227 (+ 3170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:31.731234 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:31.731248 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:31.731729 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:31.732187 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:31.732193 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:31.732197 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:31.732411 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:31.732997 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:10:31.733005 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:10:31.733343 (+ 338us) client_negotiation.cc:769] Sending connection context
0809 15:10:31.733606 (+ 263us) client_negotiation.cc:240] Negotiation successful
0809 15:10:31.734151 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":342,"threads_started":1}
I20260809 15:10:31.736925 15484 heartbeater.cc:347] Connected to a master server at 127.24.183.254:41811
I20260809 15:10:31.737247 15484 heartbeater.cc:464] Registering TS with master...
I20260809 15:10:31.738025 15484 heartbeater.cc:511] Master 127.24.183.254:41811 requested a full tablet report, sending...
I20260809 15:10:31.739605 14900 ts_manager.cc:194] Registered new tserver with Master: 0b26588483d746bdba86f18f39b0ca50 (127.24.183.194:46385)
I20260809 15:10:31.740751 14900 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.194@KRBTEST.COM'} at 127.24.183.194:48347
WARNING: no policy specified for kudu/127.24.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab.
Entry for principal kudu/127.24.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab.
I20260809 15:10:31.776352 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.195:0
--local_ip_for_outbound_sockets=127.24.183.195
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:41811
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:33207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:10:32.109647 15494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:10:32.109920 15494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:10:32.110051 15494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:10:32.117156 15494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:10:32.117252 15494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:10:32.117394 15494 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.195
I20260809 15:10:32.126809 15494 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:33207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:41811
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.195
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:10:32.128060 15494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:10:32.129405 15494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:10:32.142422 15503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:32.142511 15504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:32.142480 15507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:10:32.143548 15494 server_base.cc:1034] running on GCE node
I20260809 15:10:32.144168 15494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:10:32.145061 15494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:10:32.146252 15494 hybrid_clock.cc:648] HybridClock initialized: now 1786288232146201 us; error 48 us; skew 500 ppm
I20260809 15:10:32.146574 15494 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:10:32 dist-test-slave-nj21 krb5kdc[14825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288232, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:10:32.149789 15494 init.cc:377] Logged in from keytab as kudu/127.24.183.195@KRBTEST.COM (short username kudu)
I20260809 15:10:32.151216 15494 webserver.cc:460] Webserver started at http://127.24.183.195:46585/ using document root <none> and password file <none>
I20260809 15:10:32.151939 15494 fs_manager.cc:359] Metadata directory not provided
I20260809 15:10:32.152057 15494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:10:32.152343 15494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:10:32.155198 15494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/data/instance:
uuid: "2949328774234d318318d0202b04b980"
format_stamp: "Formatted at 2026-08-09 15:10:32 on dist-test-slave-nj21"
server_key: "ed8b8d2ec7489bf0925cadc31dd75000"
server_key_iv: "e1468b76f8d7e3bab1ad91c33934fb32"
server_key_version: "encryptionkey@0"
I20260809 15:10:32.156077 15494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance:
uuid: "2949328774234d318318d0202b04b980"
format_stamp: "Formatted at 2026-08-09 15:10:32 on dist-test-slave-nj21"
server_key: "ed8b8d2ec7489bf0925cadc31dd75000"
server_key_iv: "e1468b76f8d7e3bab1ad91c33934fb32"
server_key_version: "encryptionkey@0"
I20260809 15:10:32.161067 15494 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260809 15:10:32.164376 15517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:32.165637 15494 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:10:32.165794 15494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/wal
uuid: "2949328774234d318318d0202b04b980"
format_stamp: "Formatted at 2026-08-09 15:10:32 on dist-test-slave-nj21"
server_key: "ed8b8d2ec7489bf0925cadc31dd75000"
server_key_iv: "e1468b76f8d7e3bab1ad91c33934fb32"
server_key_version: "encryptionkey@0"
I20260809 15:10:32.166011 15494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:10:32.183266 15494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:10:32.186925 15494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:10:32.187182 15494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:10:32.188109 15494 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:10:32.189463 15494 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:10:32.189548 15494 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:32.189671 15494 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:10:32.189725 15494 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:32.223145 15494 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.195:36181
I20260809 15:10:32.223174 15713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.195:36181 every 8 connection(s)
I20260809 15:10:32.224376 15494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
I20260809 15:10:32.228404 15253 heartbeater.cc:503] Master 127.24.183.254:41811 was elected leader, sending a full tablet report...
I20260809 15:10:32.230609 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 15494
I20260809 15:10:32.230827 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance
I20260809 15:10:32.231405 25311 external_mini_cluster.cc:1428] Setting key c7a1a704ed62b1dab87687e937fd7a2a
Aug 09 15:10:32 dist-test-slave-nj21 krb5kdc[14825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288232, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:10:32.244266 15255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:32.226695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:54825 (local address 127.24.183.254:41811)
0809 15:10:32.226879 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:32.226884 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:32.228033 (+ 1149us) server_negotiation.cc:407] Connection header received
0809 15:10:32.229999 (+ 1966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:32.230004 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:32.230077 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:32.230157 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:32.232144 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:32.233239 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:32.235074 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:32.235482 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:32.235578 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:32.238965 (+ 3387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:32.238970 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:32.238974 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:32.239006 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:32.241609 (+ 2603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:32.242312 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:32.242316 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:32.242318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:32.242460 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:32.242826 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:32.242829 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:32.242831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:32.243075 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:32.243180 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:32.243847 (+ 667us) server_negotiation.cc:299] Negotiation successful
0809 15:10:32.244009 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 15:10:32.245170 15718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:32.227142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.195:54825)
0809 15:10:32.227874 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:32.227926 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:32.229722 (+ 1796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:32.230447 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:32.230458 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:32.231157 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:10:32.231976 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:32.232001 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:32.233396 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:32.233400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:32.234945 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:32.234958 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:32.235805 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:32.235812 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:32.236015 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:32.236641 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:10:32.236673 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:10:32.238778 (+ 2105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:10:32.241751 (+ 2973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:32.241757 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:32.241768 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:32.242218 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:32.242545 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:32.242548 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:32.242550 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:32.242724 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:32.243162 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:10:32.243168 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:10:32.243480 (+ 312us) client_negotiation.cc:769] Sending connection context
0809 15:10:32.243743 (+ 263us) client_negotiation.cc:240] Negotiation successful
0809 15:10:32.244093 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":300,"threads_started":1}
I20260809 15:10:32.246490 15714 heartbeater.cc:347] Connected to a master server at 127.24.183.254:41811
I20260809 15:10:32.246786 15714 heartbeater.cc:464] Registering TS with master...
I20260809 15:10:32.247506 15714 heartbeater.cc:511] Master 127.24.183.254:41811 requested a full tablet report, sending...
I20260809 15:10:32.248888 14900 ts_manager.cc:194] Registered new tserver with Master: 2949328774234d318318d0202b04b980 (127.24.183.195:36181)
I20260809 15:10:32.249684 14900 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.195@KRBTEST.COM'} at 127.24.183.195:54825
I20260809 15:10:32.258769 25311 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:10:32.276247 15255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:32.263445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56530 (local address 127.24.183.254:41811)
0809 15:10:32.263618 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:32.263626 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:32.264071 (+ 445us) server_negotiation.cc:407] Connection header received
0809 15:10:32.264251 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:32.264257 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:32.264321 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:32.264404 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:32.265936 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:32.266696 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:32.268174 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:32.268597 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:32.268748 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:32.271211 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:32.271216 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:32.271220 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:32.271252 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:32.273718 (+ 2466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:32.274353 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:32.274359 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:32.274363 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:32.274488 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:32.274861 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:32.274866 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:32.274869 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:32.275154 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:32.275391 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:32.275739 (+ 348us) server_negotiation.cc:299] Negotiation successful
0809 15:10:32.275949 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 15:10:32.279742 14900 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56530:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:10:32.286616 14900 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:10:32.311317 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:32.298996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46476 (local address 127.24.183.195:36181)
0809 15:10:32.299797 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:32.299805 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:32.300600 (+ 795us) server_negotiation.cc:407] Connection header received
0809 15:10:32.300921 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:32.300929 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:32.301185 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:32.301370 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:10:32.303430 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:32.304997 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:32.308432 (+ 3435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:32.309783 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:32.309968 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:32.310144 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:32.310800 (+ 656us) server_negotiation.cc:299] Negotiation successful
0809 15:10:32.310966 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":433,"threads_started":1}
I20260809 15:10:32.311420 15733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:32.298700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:36181 (local address 127.0.0.1:46476)
0809 15:10:32.300435 (+ 1735us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:32.300503 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:32.300683 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:32.302701 (+ 2018us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:32.302706 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:32.302736 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:10:32.303182 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:32.303195 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:32.306210 (+ 3015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:32.306216 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:32.308290 (+ 2074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:32.308300 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:32.310350 (+ 2050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:32.310357 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:32.310527 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:32.310622 (+ 95us) client_negotiation.cc:769] Sending connection context
0809 15:10:32.310733 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 15:10:32.310924 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1474,"mutex_wait_us":64,"thread_start_us":1295,"threads_started":1}
I20260809 15:10:32.311427 15736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:32.300169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49190 (local address 127.24.183.193:37317)
0809 15:10:32.301464 (+ 1295us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:32.301474 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:32.301503 (+ 29us) server_negotiation.cc:407] Connection header received
0809 15:10:32.301597 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:32.301605 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:32.301813 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:32.301997 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:10:32.303043 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:32.304674 (+ 1631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:32.308222 (+ 3548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:32.309706 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:32.309924 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:32.310114 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:32.310851 (+ 737us) server_negotiation.cc:299] Negotiation successful
0809 15:10:32.311029 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1114,"thread_start_us":1229,"threads_started":1}
I20260809 15:10:32.312101 15735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:32.299570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:37317 (local address 127.0.0.1:49190)
0809 15:10:32.300435 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:32.300503 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:32.300667 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:32.302275 (+ 1608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:32.302282 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:32.302312 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:10:32.302822 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:32.302835 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:32.304840 (+ 2005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:32.304845 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:32.308070 (+ 3225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:32.308080 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:32.310311 (+ 2231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:32.310322 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:32.310574 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:32.310627 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 15:10:32.311474 (+ 847us) client_negotiation.cc:240] Negotiation successful
0809 15:10:32.311603 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"mutex_wait_us":689,"spinlock_wait_cycles":6912,"thread_start_us":835,"threads_started":1}
I20260809 15:10:32.315634 15609 tablet_service.cc:1467] Processing CreateTablet for tablet 171ec40a212d4c228918aa5193422540 (DEFAULT_TABLE table=test-table [id=796e0218ffe3461687be37fbc225b0a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:10:32.315634 15123 tablet_service.cc:1467] Processing CreateTablet for tablet 171ec40a212d4c228918aa5193422540 (DEFAULT_TABLE table=test-table [id=796e0218ffe3461687be37fbc225b0a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:10:32.316279 15737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:32.300867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:46385 (local address 127.0.0.1:45026)
0809 15:10:32.307741 (+ 6874us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:32.307759 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:32.307870 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:32.308622 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:32.308626 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:32.308652 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:10:32.309009 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:32.309020 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:32.310897 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:32.310903 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:32.313642 (+ 2739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:32.313656 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:32.315384 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:32.315392 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:32.315570 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:32.315607 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 15:10:32.315749 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 15:10:32.315874 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6712,"thread_start_us":6569,"threads_started":1}
I20260809 15:10:32.316344 15739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:32.301216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45026 (local address 127.24.183.194:46385)
0809 15:10:32.308033 (+ 6817us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:32.308042 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:32.308070 (+ 28us) server_negotiation.cc:407] Connection header received
0809 15:10:32.308173 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:32.308181 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:32.308413 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:32.308623 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:10:32.309176 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:32.310716 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:32.313818 (+ 3102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:32.315145 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:32.315336 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:32.315512 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:32.315804 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 15:10:32.315976 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6672,"thread_start_us":6539,"threads_started":1}
I20260809 15:10:32.317245 15123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 171ec40a212d4c228918aa5193422540. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:10:32.317386 15609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 171ec40a212d4c228918aa5193422540. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:10:32.319190 15384 tablet_service.cc:1467] Processing CreateTablet for tablet 171ec40a212d4c228918aa5193422540 (DEFAULT_TABLE table=test-table [id=796e0218ffe3461687be37fbc225b0a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:10:32.320462 15384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 171ec40a212d4c228918aa5193422540. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:10:32.331740 15740 tablet_bootstrap.cc:492] T 171ec40a212d4c228918aa5193422540 P 0b26588483d746bdba86f18f39b0ca50: Bootstrap starting.
I20260809 15:10:32.333714 15742 tablet_bootstrap.cc:492] T 171ec40a212d4c228918aa5193422540 P 4e57e56718f64d37a9ee564ebad08a4e: Bootstrap starting.
I20260809 15:10:32.336419 15746 tablet_bootstrap.cc:492] T 171ec40a212d4c228918aa5193422540 P 2949328774234d318318d0202b04b980: Bootstrap starting.
I20260809 15:10:32.337369 15742 tablet_bootstrap.cc:654] T 171ec40a212d4c228918aa5193422540 P 4e57e56718f64d37a9ee564ebad08a4e: Neither blocks nor log segments found. Creating new log.
I20260809 15:10:32.335883 15740 tablet_bootstrap.cc:654] T 171ec40a212d4c228918aa5193422540 P 0b26588483d746bdba86f18f39b0ca50: Neither blocks nor log segments found. Creating new log.
I20260809 15:10:32.338768 15740 log.cc:824] T 171ec40a212d4c228918aa5193422540 P 0b26588483d746bdba86f18f39b0ca50: Log is configured to *not* fsync() on all Append() calls
I20260809 15:10:32.338778 15742 log.cc:824] T 171ec40a212d4c228918aa5193422540 P 4e57e56718f64d37a9ee564ebad08a4e: Log is configured to *not* fsync() on all Append() calls
I20260809 15:10:32.339581 15746 tablet_bootstrap.cc:654] T 171ec40a212d4c228918aa5193422540 P 2949328774234d318318d0202b04b980: Neither blocks nor log segments found. Creating new log.
I20260809 15:10:32.340796 15746 log.cc:824] T 171ec40a212d4c228918aa5193422540 P 2949328774234d318318d0202b04b980: Log is configured to *not* fsync() on all Append() calls
I20260809 15:10:32.342000 15742 tablet_bootstrap.cc:492] T 171ec40a212d4c228918aa5193422540 P 4e57e56718f64d37a9ee564ebad08a4e: No bootstrap required, opened a new log
I20260809 15:10:32.342007 15740 tablet_bootstrap.cc:492] T 171ec40a212d4c228918aa5193422540 P 0b26588483d746bdba86f18f39b0ca50: No bootstrap required, opened a new log
I20260809 15:10:32.342263 15742 ts_tablet_manager.cc:1397] T 171ec40a212d4c228918aa5193422540 P 4e57e56718f64d37a9ee564ebad08a4e: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260809 15:10:32.342272 15740 ts_tablet_manager.cc:1397] T 171ec40a212d4c228918aa5193422540 P 0b26588483d746bdba86f18f39b0ca50: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.006s
I20260809 15:10:32.343924 15746 tablet_bootstrap.cc:492] T 171ec40a212d4c228918aa5193422540 P 2949328774234d318318d0202b04b980: No bootstrap required, opened a new log
I20260809 15:10:32.344169 15746 ts_tablet_manager.cc:1397] T 171ec40a212d4c228918aa5193422540 P 2949328774234d318318d0202b04b980: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260809 15:10:32.350030 15740 raft_consensus.cc:348] T 171ec40a212d4c228918aa5193422540 P 0b26588483d746bdba86f18f39b0ca50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2949328774234d318318d0202b04b980" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36181 } } peers { permanent_uuid: "0b26588483d746bdba86f18f39b0ca50" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46385 } } peers { permanent_uuid: "4e57e56718f64d37a9ee564ebad08a4e" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 37317 } }
I20260809 15:10:32.350383 15740 raft_consensus.cc:374] T 171ec40a212d4c228918aa5193422540 P 0b26588483d746bdba86f18f39b0ca50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:10:32.350478 15740 raft_consensus.cc:729] T 171ec40a212d4c228918aa5193422540 P 0b26588483d746bdba86f18f39b0ca50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b26588483d746bdba86f18f39b0ca50, State: Initialized, Role: FOLLOWER
I20260809 15:10:32.351015 15740 consensus_queue.cc:260] T 171ec40a212d4c228918aa5193422540 P 0b26588483d746bdba86f18f39b0ca50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2949328774234d318318d0202b04b980" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36181 } } peers { permanent_uuid: "0b26588483d746bdba86f18f39b0ca50" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46385 } } peers { permanent_uuid: "4e57e56718f64d37a9ee564ebad08a4e" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 37317 } }
I20260809 15:10:32.352083 15484 heartbeater.cc:503] Master 127.24.183.254:41811 was elected leader, sending a full tablet report...
I20260809 15:10:32.352718 15740 ts_tablet_manager.cc:1428] T 171ec40a212d4c228918aa5193422540 P 0b26588483d746bdba86f18f39b0ca50: Time spent starting tablet: real 0.010s user 0.004s sys 0.007s
I20260809 15:10:32.352471 15742 raft_consensus.cc:348] T 171ec40a212d4c228918aa5193422540 P 4e57e56718f64d37a9ee564ebad08a4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2949328774234d318318d0202b04b980" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36181 } } peers { permanent_uuid: "0b26588483d746bdba86f18f39b0ca50" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46385 } } peers { permanent_uuid: "4e57e56718f64d37a9ee564ebad08a4e" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 37317 } }
I20260809 15:10:32.352861 15742 raft_consensus.cc:374] T 171ec40a212d4c228918aa5193422540 P 4e57e56718f64d37a9ee564ebad08a4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:10:32.352942 15742 raft_consensus.cc:729] T 171ec40a212d4c228918aa5193422540 P 4e57e56718f64d37a9ee564ebad08a4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e57e56718f64d37a9ee564ebad08a4e, State: Initialized, Role: FOLLOWER
I20260809 15:10:32.353492 15742 consensus_queue.cc:260] T 171ec40a212d4c228918aa5193422540 P 4e57e56718f64d37a9ee564ebad08a4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2949328774234d318318d0202b04b980" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36181 } } peers { permanent_uuid: "0b26588483d746bdba86f18f39b0ca50" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46385 } } peers { permanent_uuid: "4e57e56718f64d37a9ee564ebad08a4e" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 37317 } }
I20260809 15:10:32.354281 15746 raft_consensus.cc:348] T 171ec40a212d4c228918aa5193422540 P 2949328774234d318318d0202b04b980 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2949328774234d318318d0202b04b980" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36181 } } peers { permanent_uuid: "0b26588483d746bdba86f18f39b0ca50" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46385 } } peers { permanent_uuid: "4e57e56718f64d37a9ee564ebad08a4e" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 37317 } }
I20260809 15:10:32.354691 15746 raft_consensus.cc:374] T 171ec40a212d4c228918aa5193422540 P 2949328774234d318318d0202b04b980 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:10:32.354790 15746 raft_consensus.cc:729] T 171ec40a212d4c228918aa5193422540 P 2949328774234d318318d0202b04b980 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2949328774234d318318d0202b04b980, State: Initialized, Role: FOLLOWER
I20260809 15:10:32.355180 15742 ts_tablet_manager.cc:1428] T 171ec40a212d4c228918aa5193422540 P 4e57e56718f64d37a9ee564ebad08a4e: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
I20260809 15:10:32.355417 15746 consensus_queue.cc:260] T 171ec40a212d4c228918aa5193422540 P 2949328774234d318318d0202b04b980 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2949328774234d318318d0202b04b980" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36181 } } peers { permanent_uuid: "0b26588483d746bdba86f18f39b0ca50" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46385 } } peers { permanent_uuid: "4e57e56718f64d37a9ee564ebad08a4e" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 37317 } }
I20260809 15:10:32.357497 15714 heartbeater.cc:503] Master 127.24.183.254:41811 was elected leader, sending a full tablet report...
I20260809 15:10:32.358052 15746 ts_tablet_manager.cc:1428] T 171ec40a212d4c228918aa5193422540 P 2949328774234d318318d0202b04b980: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
W20260809 15:10:32.451153 15254 tablet.cc:2369] T 171ec40a212d4c228918aa5193422540 P 4e57e56718f64d37a9ee564ebad08a4e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:10:32.464928 15485 tablet.cc:2369] T 171ec40a212d4c228918aa5193422540 P 0b26588483d746bdba86f18f39b0ca50: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:10:32.475930 15715 tablet.cc:2369] T 171ec40a212d4c228918aa5193422540 P 2949328774234d318318d0202b04b980: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:10:32.524814 15753 raft_consensus.cc:482] T 171ec40a212d4c228918aa5193422540 P 4e57e56718f64d37a9ee564ebad08a4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:10:32.525153 15753 raft_consensus.cc:504] T 171ec40a212d4c228918aa5193422540 P 4e57e56718f64d37a9ee564ebad08a4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2949328774234d318318d0202b04b980" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36181 } } peers { permanent_uuid: "0b26588483d746bdba86f18f39b0ca50" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46385 } } peers { permanent_uuid: "4e57e56718f64d37a9ee564ebad08a4e" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 37317 } }
I20260809 15:10:32.527097 15753 leader_election.cc:288] T 171ec40a212d4c228918aa5193422540 P 4e57e56718f64d37a9ee564ebad08a4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2949328774234d318318d0202b04b980 (127.24.183.195:36181), 0b26588483d746bdba86f18f39b0ca50 (127.24.183.194:46385)
I20260809 15:10:32.534787 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:32.527608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:47991 (local address 127.24.183.195:36181)
0809 15:10:32.527861 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:32.527868 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:32.528225 (+ 357us) server_negotiation.cc:407] Connection header received
0809 15:10:32.528536 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:32.528544 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:32.528647 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:32.528760 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:10:32.529447 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:32.530467 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:32.532857 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:32.533965 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:32.534128 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:32.534205 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:32.534426 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 15:10:32.534554 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 15:10:32.534866 15756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:32.527428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:36181 (local address 127.24.183.193:47991)
0809 15:10:32.528120 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:32.528136 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:32.528302 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:32.528796 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:32.528801 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:32.528846 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:10:32.529280 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:32.529292 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:32.530597 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:32.530600 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:32.532697 (+ 2097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:32.532713 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:32.534158 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:32.534162 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:32.534280 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:32.534307 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 15:10:32.534383 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 15:10:32.534476 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":393,"threads_started":1}
I20260809 15:10:32.535337 15739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:32.528493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:60779 (local address 127.24.183.194:46385)
0809 15:10:32.528692 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:32.528697 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:32.529183 (+ 486us) server_negotiation.cc:407] Connection header received
0809 15:10:32.529406 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:32.529411 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:32.529504 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:32.529604 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:10:32.530055 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:32.530927 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:32.533285 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:32.534441 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:32.534580 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:32.534648 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:32.535039 (+ 391us) server_negotiation.cc:299] Negotiation successful
0809 15:10:32.535147 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 15:10:32.535943 15757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:32.528335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:46385 (local address 127.24.183.193:60779)
0809 15:10:32.529065 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:32.529091 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:32.529223 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:32.529615 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:32.529618 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:32.529637 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:10:32.529913 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:32.529921 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:32.531052 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:32.531058 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:32.533113 (+ 2055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:32.533124 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:32.534670 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:32.534676 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:32.534838 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:32.534875 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 15:10:32.535499 (+ 624us) client_negotiation.cc:240] Negotiation successful
0809 15:10:32.535620 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":365,"threads_started":1}
I20260809 15:10:32.536166 15637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "171ec40a212d4c228918aa5193422540" candidate_uuid: "4e57e56718f64d37a9ee564ebad08a4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2949328774234d318318d0202b04b980" is_pre_election: true
I20260809 15:10:32.536723 15637 raft_consensus.cc:2454] T 171ec40a212d4c228918aa5193422540 P 2949328774234d318318d0202b04b980 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e57e56718f64d37a9ee564ebad08a4e in term 0.
I20260809 15:10:32.536907 15420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "171ec40a212d4c228918aa5193422540" candidate_uuid: "4e57e56718f64d37a9ee564ebad08a4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b26588483d746bdba86f18f39b0ca50" is_pre_election: true
I20260809 15:10:32.537377 15048 leader_election.cc:302] T 171ec40a212d4c228918aa5193422540 P 4e57e56718f64d37a9ee564ebad08a4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2949328774234d318318d0202b04b980, 4e57e56718f64d37a9ee564ebad08a4e; no voters:
I20260809 15:10:32.537461 15420 raft_consensus.cc:2454] T 171ec40a212d4c228918aa5193422540 P 0b26588483d746bdba86f18f39b0ca50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e57e56718f64d37a9ee564ebad08a4e in term 0.
I20260809 15:10:32.537734 15753 raft_consensus.cc:2781] T 171ec40a212d4c228918aa5193422540 P 4e57e56718f64d37a9ee564ebad08a4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:10:32.537896 15753 raft_consensus.cc:482] T 171ec40a212d4c228918aa5193422540 P 4e57e56718f64d37a9ee564ebad08a4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:10:32.538025 15753 raft_consensus.cc:3037] T 171ec40a212d4c228918aa5193422540 P 4e57e56718f64d37a9ee564ebad08a4e [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:10:32.540340 15753 raft_consensus.cc:504] T 171ec40a212d4c228918aa5193422540 P 4e57e56718f64d37a9ee564ebad08a4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2949328774234d318318d0202b04b980" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36181 } } peers { permanent_uuid: "0b26588483d746bdba86f18f39b0ca50" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46385 } } peers { permanent_uuid: "4e57e56718f64d37a9ee564ebad08a4e" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 37317 } }
I20260809 15:10:32.540963 15753 leader_election.cc:288] T 171ec40a212d4c228918aa5193422540 P 4e57e56718f64d37a9ee564ebad08a4e [CANDIDATE]: Term 1 election: Requested vote from peers 2949328774234d318318d0202b04b980 (127.24.183.195:36181), 0b26588483d746bdba86f18f39b0ca50 (127.24.183.194:46385)
I20260809 15:10:32.541347 15637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "171ec40a212d4c228918aa5193422540" candidate_uuid: "4e57e56718f64d37a9ee564ebad08a4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2949328774234d318318d0202b04b980"
I20260809 15:10:32.541457 15420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "171ec40a212d4c228918aa5193422540" candidate_uuid: "4e57e56718f64d37a9ee564ebad08a4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b26588483d746bdba86f18f39b0ca50"
I20260809 15:10:32.541533 15637 raft_consensus.cc:3037] T 171ec40a212d4c228918aa5193422540 P 2949328774234d318318d0202b04b980 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:10:32.541659 15420 raft_consensus.cc:3037] T 171ec40a212d4c228918aa5193422540 P 0b26588483d746bdba86f18f39b0ca50 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:10:32.544015 15637 raft_consensus.cc:2454] T 171ec40a212d4c228918aa5193422540 P 2949328774234d318318d0202b04b980 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e57e56718f64d37a9ee564ebad08a4e in term 1.
I20260809 15:10:32.544351 15048 leader_election.cc:302] T 171ec40a212d4c228918aa5193422540 P 4e57e56718f64d37a9ee564ebad08a4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2949328774234d318318d0202b04b980, 4e57e56718f64d37a9ee564ebad08a4e; no voters:
I20260809 15:10:32.544603 15753 raft_consensus.cc:2781] T 171ec40a212d4c228918aa5193422540 P 4e57e56718f64d37a9ee564ebad08a4e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:10:32.544878 15420 raft_consensus.cc:2454] T 171ec40a212d4c228918aa5193422540 P 0b26588483d746bdba86f18f39b0ca50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e57e56718f64d37a9ee564ebad08a4e in term 1.
I20260809 15:10:32.547477 15753 raft_consensus.cc:686] T 171ec40a212d4c228918aa5193422540 P 4e57e56718f64d37a9ee564ebad08a4e [term 1 LEADER]: Becoming Leader. State: Replica: 4e57e56718f64d37a9ee564ebad08a4e, State: Running, Role: LEADER
I20260809 15:10:32.547983 15753 consensus_queue.cc:237] T 171ec40a212d4c228918aa5193422540 P 4e57e56718f64d37a9ee564ebad08a4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2949328774234d318318d0202b04b980" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36181 } } peers { permanent_uuid: "0b26588483d746bdba86f18f39b0ca50" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46385 } } peers { permanent_uuid: "4e57e56718f64d37a9ee564ebad08a4e" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 37317 } }
I20260809 15:10:32.552282 14900 catalog_manager.cc:5495] T 171ec40a212d4c228918aa5193422540 P 4e57e56718f64d37a9ee564ebad08a4e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e57e56718f64d37a9ee564ebad08a4e (127.24.183.193). New cstate: current_term: 1 leader_uuid: "4e57e56718f64d37a9ee564ebad08a4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2949328774234d318318d0202b04b980" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b26588483d746bdba86f18f39b0ca50" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e57e56718f64d37a9ee564ebad08a4e" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 37317 } health_report { overall_health: HEALTHY } } }
I20260809 15:10:32.601475 15736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:32.596100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49206 (local address 127.24.183.193:37317)
0809 15:10:32.596298 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:32.596305 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:32.596325 (+ 20us) server_negotiation.cc:407] Connection header received
0809 15:10:32.596569 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:32.596573 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:32.596651 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:32.596786 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:10:32.597187 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:32.597927 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:32.599319 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:32.599765 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:32.599888 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:32.600403 (+ 515us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:10:32.600902 (+ 499us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:10:32.601032 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:32.601131 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 15:10:32.601211 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 15:10:32.609485 15637 raft_consensus.cc:1261] T 171ec40a212d4c228918aa5193422540 P 2949328774234d318318d0202b04b980 [term 1 FOLLOWER]: Refusing update from remote peer 4e57e56718f64d37a9ee564ebad08a4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:10:32.609484 15420 raft_consensus.cc:1261] T 171ec40a212d4c228918aa5193422540 P 0b26588483d746bdba86f18f39b0ca50 [term 1 FOLLOWER]: Refusing update from remote peer 4e57e56718f64d37a9ee564ebad08a4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:10:32.610368 15753 consensus_queue.cc:1035] T 171ec40a212d4c228918aa5193422540 P 4e57e56718f64d37a9ee564ebad08a4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b26588483d746bdba86f18f39b0ca50" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:10:32.610680 15758 consensus_queue.cc:1035] T 171ec40a212d4c228918aa5193422540 P 4e57e56718f64d37a9ee564ebad08a4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2949328774234d318318d0202b04b980" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:10:32.621541 15763 mvcc.cc:205] Tried to move back new op lower bound from 7316636600758489088 to 7316636600519921664. Current Snapshot: MvccSnapshot[applied={T|T < 7316636600758489088}]
I20260809 15:10:32.627445 15764 mvcc.cc:205] Tried to move back new op lower bound from 7316636600758489088 to 7316636600519921664. Current Snapshot: MvccSnapshot[applied={T|T < 7316636600758489088}]
I20260809 15:10:32.635143 14900 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:10:32.635372 14900 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:10:32.638785 14900 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951: Sending DeleteTablet for 3 replicas of tablet 171ec40a212d4c228918aa5193422540
I20260809 15:10:32.639840 15609 tablet_service.cc:1514] Processing DeleteTablet for tablet 171ec40a212d4c228918aa5193422540 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:10:32 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:46476
I20260809 15:10:32.639900 15123 tablet_service.cc:1514] Processing DeleteTablet for tablet 171ec40a212d4c228918aa5193422540 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:10:32 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:49190
I20260809 15:10:32.639916 15384 tablet_service.cc:1514] Processing DeleteTablet for tablet 171ec40a212d4c228918aa5193422540 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:10:32 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:45026
I20260809 15:10:32.644775 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 15017
I20260809 15:10:32.652264 15774 tablet_replica.cc:330] T 171ec40a212d4c228918aa5193422540 P 2949328774234d318318d0202b04b980: stopping tablet replica
I20260809 15:10:32.652339 15778 tablet_replica.cc:330] T 171ec40a212d4c228918aa5193422540 P 4e57e56718f64d37a9ee564ebad08a4e: stopping tablet replica
I20260809 15:10:32.652803 15778 raft_consensus.cc:2229] T 171ec40a212d4c228918aa5193422540 P 4e57e56718f64d37a9ee564ebad08a4e [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:10:32.652858 15774 raft_consensus.cc:2229] T 171ec40a212d4c228918aa5193422540 P 2949328774234d318318d0202b04b980 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:10:32.653465 15774 raft_consensus.cc:2258] T 171ec40a212d4c228918aa5193422540 P 2949328774234d318318d0202b04b980 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:10:32.653503 15778 raft_consensus.cc:2258] T 171ec40a212d4c228918aa5193422540 P 4e57e56718f64d37a9ee564ebad08a4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:10:32.655766 15774 ts_tablet_manager.cc:1905] T 171ec40a212d4c228918aa5193422540 P 2949328774234d318318d0202b04b980: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:10:32.658778 15778 ts_tablet_manager.cc:1905] T 171ec40a212d4c228918aa5193422540 P 4e57e56718f64d37a9ee564ebad08a4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:10:32.658965 15777 tablet_replica.cc:330] T 171ec40a212d4c228918aa5193422540 P 0b26588483d746bdba86f18f39b0ca50: stopping tablet replica
I20260809 15:10:32.659350 15777 raft_consensus.cc:2229] T 171ec40a212d4c228918aa5193422540 P 0b26588483d746bdba86f18f39b0ca50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:10:32.659895 15777 raft_consensus.cc:2258] T 171ec40a212d4c228918aa5193422540 P 0b26588483d746bdba86f18f39b0ca50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:10:32.661967 15777 ts_tablet_manager.cc:1905] T 171ec40a212d4c228918aa5193422540 P 0b26588483d746bdba86f18f39b0ca50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:10:32.666106 15778 ts_tablet_manager.cc:1918] T 171ec40a212d4c228918aa5193422540 P 4e57e56718f64d37a9ee564ebad08a4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:10:32.666276 15778 log.cc:1196] T 171ec40a212d4c228918aa5193422540 P 4e57e56718f64d37a9ee564ebad08a4e: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/wal/wals/171ec40a212d4c228918aa5193422540
I20260809 15:10:32.666867 15778 ts_tablet_manager.cc:1939] T 171ec40a212d4c228918aa5193422540 P 4e57e56718f64d37a9ee564ebad08a4e: Deleting consensus metadata
I20260809 15:10:32.668424 15777 ts_tablet_manager.cc:1918] T 171ec40a212d4c228918aa5193422540 P 0b26588483d746bdba86f18f39b0ca50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:10:32.668582 15777 log.cc:1196] T 171ec40a212d4c228918aa5193422540 P 0b26588483d746bdba86f18f39b0ca50: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/wal/wals/171ec40a212d4c228918aa5193422540
I20260809 15:10:32.668952 14867 catalog_manager.cc:4841] TS 4e57e56718f64d37a9ee564ebad08a4e (127.24.183.193:37317): tablet 171ec40a212d4c228918aa5193422540 (table test-table [id=796e0218ffe3461687be37fbc225b0a6]) successfully deleted
I20260809 15:10:32.669178 15777 ts_tablet_manager.cc:1939] T 171ec40a212d4c228918aa5193422540 P 0b26588483d746bdba86f18f39b0ca50: Deleting consensus metadata
I20260809 15:10:32.669469 15774 ts_tablet_manager.cc:1918] T 171ec40a212d4c228918aa5193422540 P 2949328774234d318318d0202b04b980: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:10:32.669639 15774 log.cc:1196] T 171ec40a212d4c228918aa5193422540 P 2949328774234d318318d0202b04b980: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/wal/wals/171ec40a212d4c228918aa5193422540
I20260809 15:10:32.670140 15774 ts_tablet_manager.cc:1939] T 171ec40a212d4c228918aa5193422540 P 2949328774234d318318d0202b04b980: Deleting consensus metadata
I20260809 15:10:32.670936 14871 catalog_manager.cc:4841] TS 0b26588483d746bdba86f18f39b0ca50 (127.24.183.194:46385): tablet 171ec40a212d4c228918aa5193422540 (table test-table [id=796e0218ffe3461687be37fbc225b0a6]) successfully deleted
I20260809 15:10:32.671752 14871 catalog_manager.cc:4841] TS 2949328774234d318318d0202b04b980 (127.24.183.195:36181): tablet 171ec40a212d4c228918aa5193422540 (table test-table [id=796e0218ffe3461687be37fbc225b0a6]) successfully deleted
Aug 09 15:10:32 dist-test-slave-nj21 krb5kdc[14825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.193@KRBTEST.COM
I20260809 15:10:32.683495 15736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:32.645936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49208 (local address 127.24.183.193:37317)
0809 15:10:32.646113 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:32.646120 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:32.658777 (+ 12657us) server_negotiation.cc:407] Connection header received
0809 15:10:32.659797 (+ 1020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:32.659803 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:32.659882 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:32.659963 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:32.666340 (+ 6377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:32.667631 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:32.670416 (+ 2785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:32.671161 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:32.671259 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:32.676727 (+ 5468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:32.676742 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:32.676762 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:32.676816 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:32.680530 (+ 3714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:32.681209 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:32.681223 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:32.681227 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:32.681445 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:32.681818 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:32.681824 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:32.681827 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:32.682286 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:32.682603 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:32.682848 (+ 245us) server_negotiation.cc:299] Negotiation successful
0809 15:10:32.683094 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 15:10:32.684231 15247 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:10:32.798617 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 15017
I20260809 15:10:32.819659 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 15260
Aug 09 15:10:32 dist-test-slave-nj21 krb5kdc[14825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.194@KRBTEST.COM
I20260809 15:10:32.834220 15739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:32.820424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45038 (local address 127.24.183.194:46385)
0809 15:10:32.820580 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:32.820585 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:32.820606 (+ 21us) server_negotiation.cc:407] Connection header received
0809 15:10:32.820767 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:32.820771 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:32.820847 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:32.820930 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:32.822131 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:32.822873 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:32.824273 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:32.824650 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:32.824743 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:32.829147 (+ 4404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:32.829162 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:32.829177 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:32.829211 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:32.831942 (+ 2731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:32.832560 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:32.832567 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:32.832569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:32.832686 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:32.832946 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:32.832948 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:32.832950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:32.833318 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:32.833550 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:32.833752 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 15:10:32.833934 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 15:10:32.834900 15477 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:10:32.924724 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 15260
I20260809 15:10:32.948868 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 15494
Aug 09 15:10:32 dist-test-slave-nj21 krb5kdc[14825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.195@KRBTEST.COM
I20260809 15:10:32.963655 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:32.949614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46490 (local address 127.24.183.195:36181)
0809 15:10:32.949782 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:32.949787 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:32.949806 (+ 19us) server_negotiation.cc:407] Connection header received
0809 15:10:32.949938 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:32.949943 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:32.950011 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:32.950106 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:32.951401 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:32.952217 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:32.953496 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:32.953929 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:32.954029 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:32.958447 (+ 4418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:32.958464 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:32.958483 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:32.958519 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:32.961150 (+ 2631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:32.961821 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:32.961829 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:32.961831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:32.961987 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:32.962304 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:32.962307 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:32.962309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:32.962697 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:32.962939 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:32.963120 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 15:10:32.963296 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 15:10:32.964511 15702 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:10:33.052886 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 15494
I20260809 15:10:33.077452 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 14845
I20260809 15:10:33.078776 14959 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:10:33.192037 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 14845
I20260809 15:10:33.218602 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:41811
--webserver_interface=127.24.183.254
--webserver_port=41829
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:33207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:41811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:10:33.551162 15788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:10:33.551437 15788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:10:33.551563 15788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:10:33.558615 15788 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:10:33.558717 15788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:10:33.558792 15788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:10:33.558871 15788 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:10:33.558931 15788 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:10:33.568624 15788 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:33207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:41811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:41811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=41829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:10:33.569914 15788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:10:33.571180 15788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:10:33.597157 15809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:33.597114 15806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:33.597185 15801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:10:33.598038 15788 server_base.cc:1034] running on GCE node
I20260809 15:10:33.598914 15788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:10:33.600520 15788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:10:33.601724 15788 hybrid_clock.cc:648] HybridClock initialized: now 1786288233601671 us; error 40 us; skew 500 ppm
I20260809 15:10:33.602126 15788 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:10:33 dist-test-slave-nj21 krb5kdc[14825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288233, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:10:33.605626 15788 init.cc:377] Logged in from keytab as kudu/127.24.183.254@KRBTEST.COM (short username kudu)
I20260809 15:10:33.607349 15788 webserver.cc:460] Webserver started at http://127.24.183.254:41829/ using document root <none> and password file <none>
I20260809 15:10:33.608144 15788 fs_manager.cc:359] Metadata directory not provided
I20260809 15:10:33.608283 15788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:10:33.614267 15788 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.001s
I20260809 15:10:33.617161 15822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:33.618462 15788 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 15:10:33.618625 15788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "49997ba886fa4ba9b145e19999ab8951"
format_stamp: "Formatted at 2026-08-09 15:10:30 on dist-test-slave-nj21"
server_key: "90039f93bb59fa1a23ff0e4d8ba72425"
server_key_iv: "b1505f0f0c2c6d93cd9902ca659cdeb1"
server_key_version: "encryptionkey@0"
I20260809 15:10:33.619714 15788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:10:33.648772 15788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:10:33.652716 15788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:10:33.652964 15788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:10:33.671067 15788 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:41811
I20260809 15:10:33.671074 15919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:41811 every 8 connection(s)
I20260809 15:10:33.672529 15788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:10:33.673411 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 15788
I20260809 15:10:33.674270 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.193:37317
--local_ip_for_outbound_sockets=127.24.183.193
--tserver_master_addrs=127.24.183.254:41811
--webserver_port=35243
--webserver_interface=127.24.183.193
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:33207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:10:33.681159 15920 sys_catalog.cc:263] Verifying existing consensus state
I20260809 15:10:33.685173 15920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951: Bootstrap starting.
I20260809 15:10:33.711450 15920 log.cc:824] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951: Log is configured to *not* fsync() on all Append() calls
I20260809 15:10:33.727239 15920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:10:33.727921 15920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951: Bootstrap complete.
I20260809 15:10:33.741451 15920 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49997ba886fa4ba9b145e19999ab8951" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 41811 } }
I20260809 15:10:33.742596 15920 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49997ba886fa4ba9b145e19999ab8951, State: Initialized, Role: FOLLOWER
I20260809 15:10:33.743306 15920 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49997ba886fa4ba9b145e19999ab8951" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 41811 } }
I20260809 15:10:33.743597 15920 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:10:33.743759 15920 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:10:33.743948 15920 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:10:33.749887 15920 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49997ba886fa4ba9b145e19999ab8951" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 41811 } }
I20260809 15:10:33.750563 15920 leader_election.cc:302] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49997ba886fa4ba9b145e19999ab8951; no voters:
I20260809 15:10:33.751307 15920 leader_election.cc:288] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 15:10:33.751430 15927 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:10:33.752635 15927 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [term 2 LEADER]: Becoming Leader. State: Replica: 49997ba886fa4ba9b145e19999ab8951, State: Running, Role: LEADER
I20260809 15:10:33.753149 15927 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49997ba886fa4ba9b145e19999ab8951" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 41811 } }
I20260809 15:10:33.753543 15920 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:10:33.755481 15931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49997ba886fa4ba9b145e19999ab8951. Latest consensus state: current_term: 2 leader_uuid: "49997ba886fa4ba9b145e19999ab8951" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49997ba886fa4ba9b145e19999ab8951" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 41811 } } }
I20260809 15:10:33.756294 15931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [sys.catalog]: This master's current role is: LEADER
I20260809 15:10:33.755820 15929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "49997ba886fa4ba9b145e19999ab8951" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49997ba886fa4ba9b145e19999ab8951" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 41811 } } }
I20260809 15:10:33.756868 15929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [sys.catalog]: This master's current role is: LEADER
I20260809 15:10:33.758440 15943 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:10:33.767284 15943 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:10:33.768677 15943 catalog_manager.cc:1259] Loaded cluster ID: 67744b2672344797b8ec69afdbdf09a1
I20260809 15:10:33.768761 15943 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:10:33.772240 15943 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 15:10:33.795791 15943 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 15:10:33.805176 15943 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:10:33.806931 15943 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951: Loaded TSK: 0
I20260809 15:10:33.808463 15943 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 15:10:34.050869 15923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:10:34.051151 15923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:10:34.051294 15923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:10:34.058568 15923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:10:34.058681 15923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:10:34.058842 15923 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.193
I20260809 15:10:34.068616 15923 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:33207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.193:37317
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.183.193
--webserver_port=35243
--tserver_master_addrs=127.24.183.254:41811
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.193
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:10:34.069991 15923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:10:34.071326 15923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:10:34.086084 15974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:34.086717 15976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:34.086081 15966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:10:34.089002 15923 server_base.cc:1034] running on GCE node
I20260809 15:10:34.089613 15923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:10:34.090565 15923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:10:34.091797 15923 hybrid_clock.cc:648] HybridClock initialized: now 1786288234091733 us; error 43 us; skew 500 ppm
I20260809 15:10:34.092119 15923 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:10:34 dist-test-slave-nj21 krb5kdc[14825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288234, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:10:34.095376 15923 init.cc:377] Logged in from keytab as kudu/127.24.183.193@KRBTEST.COM (short username kudu)
I20260809 15:10:34.097024 15923 webserver.cc:460] Webserver started at http://127.24.183.193:35243/ using document root <none> and password file <none>
I20260809 15:10:34.097738 15923 fs_manager.cc:359] Metadata directory not provided
I20260809 15:10:34.097888 15923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:10:34.103425 15923 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 15:10:34.106129 15985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:34.107486 15923 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 15:10:34.107625 15923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/wal
uuid: "4e57e56718f64d37a9ee564ebad08a4e"
format_stamp: "Formatted at 2026-08-09 15:10:31 on dist-test-slave-nj21"
server_key: "cf13abac65bace41da3d311e65d63473"
server_key_iv: "74bb9cf8d79954e751ec3d9cb04aab06"
server_key_version: "encryptionkey@0"
I20260809 15:10:34.108677 15923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:10:34.142647 15923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:10:34.146811 15923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:10:34.147080 15923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:10:34.148149 15923 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:10:34.149560 15923 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:10:34.149653 15923 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:34.149760 15923 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:10:34.149829 15923 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:34.181201 15923 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.193:37317
I20260809 15:10:34.181270 16177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.193:37317 every 8 connection(s)
I20260809 15:10:34.182663 15923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
I20260809 15:10:34.188503 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 15923
I20260809 15:10:34.189270 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.194:46385
--local_ip_for_outbound_sockets=127.24.183.194
--tserver_master_addrs=127.24.183.254:41811
--webserver_port=33949
--webserver_interface=127.24.183.194
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:33207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 15:10:34 dist-test-slave-nj21 krb5kdc[14825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288234, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:10:34.206369 16186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:34.185403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:53073 (local address 127.24.183.254:41811)
0809 15:10:34.185969 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:34.185981 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:34.186681 (+ 700us) server_negotiation.cc:407] Connection header received
0809 15:10:34.188960 (+ 2279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:34.188980 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:34.189404 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:34.189806 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:34.191156 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:34.192995 (+ 1839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:34.194666 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:34.195596 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:34.195801 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:34.199274 (+ 3473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:34.199283 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:34.199302 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:34.199351 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:34.202331 (+ 2980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:34.203155 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:34.203162 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:34.203167 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:34.203328 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:34.203765 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:34.203768 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:34.203770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:34.204207 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:34.204356 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:34.205153 (+ 797us) server_negotiation.cc:299] Negotiation successful
0809 15:10:34.205450 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":244,"threads_started":1}
I20260809 15:10:34.206632 16188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:34.185683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.193:53073)
0809 15:10:34.186519 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:34.186572 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:34.188366 (+ 1794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:34.189923 (+ 1557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:34.189931 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:34.190411 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:10:34.190932 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:34.190949 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:34.193203 (+ 2254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:34.193208 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:34.194514 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:34.194530 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:34.195855 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:34.195861 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:34.196052 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:34.196671 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:10:34.196697 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:10:34.198992 (+ 2295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:10:34.202470 (+ 3478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:34.202478 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:34.202492 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:34.203048 (+ 556us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:34.203430 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:34.203434 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:34.203437 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:34.203656 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:34.204342 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:10:34.204350 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:10:34.204710 (+ 360us) client_negotiation.cc:769] Sending connection context
0809 15:10:34.204990 (+ 280us) client_negotiation.cc:240] Negotiation successful
0809 15:10:34.205514 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":341,"threads_started":1}
I20260809 15:10:34.209674 16184 heartbeater.cc:347] Connected to a master server at 127.24.183.254:41811
I20260809 15:10:34.209954 16184 heartbeater.cc:464] Registering TS with master...
I20260809 15:10:34.210655 16184 heartbeater.cc:511] Master 127.24.183.254:41811 requested a full tablet report, sending...
I20260809 15:10:34.213052 15855 ts_manager.cc:194] Registered new tserver with Master: 4e57e56718f64d37a9ee564ebad08a4e (127.24.183.193:37317)
I20260809 15:10:34.215095 15855 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.193@KRBTEST.COM'} at 127.24.183.193:53073
W20260809 15:10:34.524686 16189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:10:34.524950 16189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:10:34.525054 16189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:10:34.531970 16189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:10:34.532068 16189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:10:34.532177 16189 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.194
I20260809 15:10:34.541494 16189 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:33207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.194:46385
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.183.194
--webserver_port=33949
--tserver_master_addrs=127.24.183.254:41811
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.194
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:10:34.542661 16189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:10:34.543984 16189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:10:34.556406 16200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:34.556437 16198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:34.556717 16203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:10:34.557351 16189 server_base.cc:1034] running on GCE node
I20260809 15:10:34.557905 16189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:10:34.558837 16189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:10:34.560047 16189 hybrid_clock.cc:648] HybridClock initialized: now 1786288234559997 us; error 48 us; skew 500 ppm
I20260809 15:10:34.560344 16189 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:10:34 dist-test-slave-nj21 krb5kdc[14825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288234, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:10:34.563388 16189 init.cc:377] Logged in from keytab as kudu/127.24.183.194@KRBTEST.COM (short username kudu)
I20260809 15:10:34.564783 16189 webserver.cc:460] Webserver started at http://127.24.183.194:33949/ using document root <none> and password file <none>
I20260809 15:10:34.565447 16189 fs_manager.cc:359] Metadata directory not provided
I20260809 15:10:34.565578 16189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:10:34.571020 16189 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260809 15:10:34.573715 16211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:34.574996 16189 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 15:10:34.575150 16189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/wal
uuid: "0b26588483d746bdba86f18f39b0ca50"
format_stamp: "Formatted at 2026-08-09 15:10:31 on dist-test-slave-nj21"
server_key: "0daae33d260d9132d5fb962faa6f6c75"
server_key_iv: "98aa459986d440a9d2a7d13bfa7fc021"
server_key_version: "encryptionkey@0"
I20260809 15:10:34.576131 16189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:10:34.591223 16189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:10:34.594937 16189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:10:34.595173 16189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:10:34.596189 16189 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:10:34.597473 16189 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:10:34.597577 16189 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:34.597682 16189 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:10:34.597762 16189 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:34.628638 16189 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.194:46385
I20260809 15:10:34.628703 16407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.194:46385 every 8 connection(s)
I20260809 15:10:34.630072 16189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
I20260809 15:10:34.638963 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 16189
I20260809 15:10:34.639549 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.195:36181
--local_ip_for_outbound_sockets=127.24.183.195
--tserver_master_addrs=127.24.183.254:41811
--webserver_port=46585
--webserver_interface=127.24.183.195
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:33207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 15:10:34 dist-test-slave-nj21 krb5kdc[14825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288234, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:10:34.649878 16186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:34.632594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:49321 (local address 127.24.183.254:41811)
0809 15:10:34.632799 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:34.632805 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:34.633991 (+ 1186us) server_negotiation.cc:407] Connection header received
0809 15:10:34.635800 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:34.635805 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:34.635877 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:34.635961 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:34.637414 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:34.638179 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:34.639819 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:34.640264 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:34.640351 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:34.643984 (+ 3633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:34.643990 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:34.643995 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:34.644036 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:34.646888 (+ 2852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:34.647595 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:34.647600 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:34.647603 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:34.647778 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:34.648158 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:34.648162 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:34.648165 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:34.648499 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:34.648646 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:34.649292 (+ 646us) server_negotiation.cc:299] Negotiation successful
0809 15:10:34.649544 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 15:10:34.650738 16410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:34.633093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.194:49321)
0809 15:10:34.633830 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:34.633876 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:34.635513 (+ 1637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:34.636149 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:34.636158 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:34.636677 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:10:34.637232 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:34.637250 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:34.638336 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:34.638341 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:34.639611 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:34.639627 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:34.640627 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:34.640632 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:34.640835 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:34.641469 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:10:34.641491 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:10:34.643774 (+ 2283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:10:34.647031 (+ 3257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:34.647038 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:34.647051 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:34.647471 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:34.647868 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:34.647871 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:34.647873 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:34.648054 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:34.648601 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:10:34.648607 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:10:34.648917 (+ 310us) client_negotiation.cc:769] Sending connection context
0809 15:10:34.649142 (+ 225us) client_negotiation.cc:240] Negotiation successful
0809 15:10:34.649550 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":271,"threads_started":1}
I20260809 15:10:34.652299 16408 heartbeater.cc:347] Connected to a master server at 127.24.183.254:41811
I20260809 15:10:34.652601 16408 heartbeater.cc:464] Registering TS with master...
I20260809 15:10:34.653337 16408 heartbeater.cc:511] Master 127.24.183.254:41811 requested a full tablet report, sending...
I20260809 15:10:34.654909 15855 ts_manager.cc:194] Registered new tserver with Master: 0b26588483d746bdba86f18f39b0ca50 (127.24.183.194:46385)
I20260809 15:10:34.656003 15855 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.194@KRBTEST.COM'} at 127.24.183.194:49321
W20260809 15:10:34.975767 16411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:10:34.976094 16411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:10:34.976199 16411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:10:34.984884 16411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:10:34.985008 16411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:10:34.985128 16411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.195
I20260809 15:10:34.995457 16411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:33207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.195:36181
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.183.195
--webserver_port=46585
--tserver_master_addrs=127.24.183.254:41811
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.195
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:10:34.996884 16411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:10:34.998251 16411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:10:35.011931 16433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:35.011967 16429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:35.011889 16430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:10:35.012776 16411 server_base.cc:1034] running on GCE node
I20260809 15:10:35.013389 16411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:10:35.014398 16411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:10:35.015609 16411 hybrid_clock.cc:648] HybridClock initialized: now 1786288235015552 us; error 50 us; skew 500 ppm
I20260809 15:10:35.015944 16411 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:10:35 dist-test-slave-nj21 krb5kdc[14825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288235, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:10:35.019151 16411 init.cc:377] Logged in from keytab as kudu/127.24.183.195@KRBTEST.COM (short username kudu)
I20260809 15:10:35.020617 16411 webserver.cc:460] Webserver started at http://127.24.183.195:46585/ using document root <none> and password file <none>
I20260809 15:10:35.021294 16411 fs_manager.cc:359] Metadata directory not provided
I20260809 15:10:35.021430 16411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:10:35.026958 16411 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260809 15:10:35.029708 16442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:35.031077 16411 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:10:35.031244 16411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/wal
uuid: "2949328774234d318318d0202b04b980"
format_stamp: "Formatted at 2026-08-09 15:10:32 on dist-test-slave-nj21"
server_key: "ed8b8d2ec7489bf0925cadc31dd75000"
server_key_iv: "e1468b76f8d7e3bab1ad91c33934fb32"
server_key_version: "encryptionkey@0"
I20260809 15:10:35.032277 16411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:10:35.056272 16411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:10:35.060236 16411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:10:35.060510 16411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:10:35.061540 16411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:10:35.062906 16411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:10:35.062997 16411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:35.063103 16411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:10:35.063174 16411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:35.094502 16411 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.195:36181
I20260809 15:10:35.094560 16630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.195:36181 every 8 connection(s)
I20260809 15:10:35.095955 16411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
I20260809 15:10:35.098670 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 16411
Aug 09 15:10:35 dist-test-slave-nj21 krb5kdc[14825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288235, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:10:35.117763 16186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:35.098512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:37359 (local address 127.24.183.254:41811)
0809 15:10:35.098730 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:35.098737 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:35.099976 (+ 1239us) server_negotiation.cc:407] Connection header received
0809 15:10:35.101879 (+ 1903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:35.101884 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:35.101965 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:35.102052 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:35.103498 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:35.104289 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:35.106470 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:35.106899 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:35.106993 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:35.111211 (+ 4218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:35.111223 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:35.111228 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:35.111297 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:35.114555 (+ 3258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:35.115397 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:35.115401 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:35.115403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:35.115523 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:35.116010 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:35.116013 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:35.116015 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:35.116361 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:35.116481 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:35.117311 (+ 830us) server_negotiation.cc:299] Negotiation successful
0809 15:10:35.117510 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76,"spinlock_wait_cycles":29312}
I20260809 15:10:35.118904 16633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:35.098998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.195:37359)
0809 15:10:35.099824 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:35.099869 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:35.101608 (+ 1739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:35.102296 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:35.102304 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:35.102796 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:10:35.103329 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:35.103346 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:35.104447 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:35.104452 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:35.106309 (+ 1857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:35.106326 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:35.107131 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:35.107138 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:35.107425 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:35.108205 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:10:35.108244 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:10:35.110987 (+ 2743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:10:35.114682 (+ 3695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:35.114692 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:35.114707 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:35.115271 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:35.115633 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:35.115637 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:35.115641 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:35.115901 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:35.116491 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:10:35.116499 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:10:35.116921 (+ 422us) client_negotiation.cc:769] Sending connection context
0809 15:10:35.117219 (+ 298us) client_negotiation.cc:240] Negotiation successful
0809 15:10:35.117746 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":318,"threads_started":1}
I20260809 15:10:35.120452 16631 heartbeater.cc:347] Connected to a master server at 127.24.183.254:41811
I20260809 15:10:35.120499 16635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:35.099517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34620 (local address 127.24.183.254:41811)
0809 15:10:35.104755 (+ 5238us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:35.104761 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:35.105292 (+ 531us) server_negotiation.cc:407] Connection header received
0809 15:10:35.105468 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:35.105471 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:35.105538 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:35.105626 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:35.107401 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:35.108392 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:35.110283 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:35.110987 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:35.111120 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:35.114499 (+ 3379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:35.114504 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:35.114508 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:35.114538 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:35.117801 (+ 3263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:35.118617 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:35.118622 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:35.118625 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:35.118765 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:35.119125 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:35.119130 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:35.119133 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:35.119476 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:35.119759 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:35.120030 (+ 271us) server_negotiation.cc:299] Negotiation successful
0809 15:10:35.120194 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5132,"spinlock_wait_cycles":9600,"thread_start_us":4999,"threads_started":1}
I20260809 15:10:35.120788 16631 heartbeater.cc:464] Registering TS with master...
I20260809 15:10:35.121706 16631 heartbeater.cc:511] Master 127.24.183.254:41811 requested a full tablet report, sending...
I20260809 15:10:35.123148 15855 ts_manager.cc:194] Registered new tserver with Master: 2949328774234d318318d0202b04b980 (127.24.183.195:36181)
I20260809 15:10:35.124023 15855 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.195@KRBTEST.COM'} at 127.24.183.195:37359
I20260809 15:10:35.134874 25311 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:10:35.141608 16635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:35.135895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34624 (local address 127.24.183.254:41811)
0809 15:10:35.136059 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:35.136065 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:35.136515 (+ 450us) server_negotiation.cc:407] Connection header received
0809 15:10:35.136712 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:35.136716 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:35.136909 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:35.137033 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:10:35.137392 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:35.138079 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:35.139574 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:35.140083 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:35.140172 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:35.140677 (+ 505us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:10:35.141159 (+ 482us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:10:35.141266 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:35.141354 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 15:10:35.141415 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 15:10:35.142439 15855 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:34624:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:10:35.149763 15855 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:10:35.172048 16651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:35.161151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:46385 (local address 127.0.0.1:50882)
0809 15:10:35.162117 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:35.162160 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:35.162357 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:35.163608 (+ 1251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:35.163615 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:35.163650 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:10:35.164190 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:35.164206 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:35.166291 (+ 2085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:35.166297 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:35.168918 (+ 2621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:35.168935 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:35.170997 (+ 2062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:35.171004 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:35.171217 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:35.171274 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 15:10:35.171401 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 15:10:35.171596 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":710,"thread_start_us":2689,"threads_started":1}
I20260809 15:10:35.173411 16652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:35.161411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50882 (local address 127.24.183.194:46385)
0809 15:10:35.162998 (+ 1587us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:35.163007 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:35.163037 (+ 30us) server_negotiation.cc:407] Connection header received
0809 15:10:35.163132 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:35.163139 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:35.163382 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:35.163578 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:10:35.164390 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:35.166100 (+ 1710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:35.169151 (+ 3051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:35.170714 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:35.172564 (+ 1850us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:35.172771 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:35.172871 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 15:10:35.173055 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1443,"thread_start_us":1273,"threads_started":1}
I20260809 15:10:35.174297 16654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:35.161942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47772 (local address 127.24.183.193:37317)
0809 15:10:35.163251 (+ 1309us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:35.163259 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:35.164130 (+ 871us) server_negotiation.cc:407] Connection header received
0809 15:10:35.164379 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:35.164386 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:35.164583 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:35.164750 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:10:35.165755 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:35.167260 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:35.171233 (+ 3973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:35.172799 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:35.172955 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:35.173122 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:35.173765 (+ 643us) server_negotiation.cc:299] Negotiation successful
0809 15:10:35.173951 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1166,"thread_start_us":1056,"threads_started":1}
I20260809 15:10:35.175009 16650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:35.161352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:37317 (local address 127.0.0.1:47772)
0809 15:10:35.164022 (+ 2670us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:35.164038 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:35.164165 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:35.165034 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:35.165040 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:35.165068 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:10:35.165551 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:35.165564 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:35.167474 (+ 1910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:35.167480 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:35.171045 (+ 3565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:35.171061 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:35.173359 (+ 2298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:35.173366 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:35.173549 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:35.173579 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 15:10:35.174505 (+ 926us) client_negotiation.cc:240] Negotiation successful
0809 15:10:35.174652 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2568,"thread_start_us":867,"threads_started":1}
I20260809 15:10:35.179394 16294 tablet_service.cc:1467] Processing CreateTablet for tablet 837cbee498114752a8f0f2b7a1557570 (DEFAULT_TABLE table=test-table [id=7554ecbce8d6404b8a85c91b837f7a71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:10:35.179378 16065 tablet_service.cc:1467] Processing CreateTablet for tablet 837cbee498114752a8f0f2b7a1557570 (DEFAULT_TABLE table=test-table [id=7554ecbce8d6404b8a85c91b837f7a71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:10:35.180882 16294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 837cbee498114752a8f0f2b7a1557570. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:10:35.180931 16065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 837cbee498114752a8f0f2b7a1557570. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:10:35.183079 16651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:35.164240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:36181 (local address 127.0.0.1:56988)
0809 15:10:35.172267 (+ 8027us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:35.172283 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:35.172371 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:35.176639 (+ 4268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:35.176644 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:35.176671 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:10:35.176839 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:35.176850 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:35.178733 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:35.178739 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:35.180497 (+ 1758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:35.180522 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:35.182374 (+ 1852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:35.182380 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:35.182569 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:35.182603 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 15:10:35.182673 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 15:10:35.182811 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7946,"thread_start_us":11890,"threads_started":1}
I20260809 15:10:35.184226 16656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:35.164538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56988 (local address 127.24.183.195:36181)
0809 15:10:35.176032 (+ 11494us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:35.176042 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:35.176078 (+ 36us) server_negotiation.cc:407] Connection header received
0809 15:10:35.176185 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:35.176194 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:35.176444 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:35.176650 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:10:35.177021 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:35.178569 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:35.180791 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:35.182168 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:35.183486 (+ 1318us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:35.183658 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:35.183746 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 15:10:35.183905 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11324,"thread_start_us":11173,"threads_started":1}
I20260809 15:10:35.187517 16524 tablet_service.cc:1467] Processing CreateTablet for tablet 837cbee498114752a8f0f2b7a1557570 (DEFAULT_TABLE table=test-table [id=7554ecbce8d6404b8a85c91b837f7a71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:10:35.188642 16524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 837cbee498114752a8f0f2b7a1557570. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:10:35.198336 16657 tablet_bootstrap.cc:492] T 837cbee498114752a8f0f2b7a1557570 P 4e57e56718f64d37a9ee564ebad08a4e: Bootstrap starting.
I20260809 15:10:35.201578 16657 tablet_bootstrap.cc:654] T 837cbee498114752a8f0f2b7a1557570 P 4e57e56718f64d37a9ee564ebad08a4e: Neither blocks nor log segments found. Creating new log.
I20260809 15:10:35.203094 16660 tablet_bootstrap.cc:492] T 837cbee498114752a8f0f2b7a1557570 P 2949328774234d318318d0202b04b980: Bootstrap starting.
I20260809 15:10:35.203192 16657 log.cc:824] T 837cbee498114752a8f0f2b7a1557570 P 4e57e56718f64d37a9ee564ebad08a4e: Log is configured to *not* fsync() on all Append() calls
I20260809 15:10:35.204713 16661 tablet_bootstrap.cc:492] T 837cbee498114752a8f0f2b7a1557570 P 0b26588483d746bdba86f18f39b0ca50: Bootstrap starting.
I20260809 15:10:35.206288 16657 tablet_bootstrap.cc:492] T 837cbee498114752a8f0f2b7a1557570 P 4e57e56718f64d37a9ee564ebad08a4e: No bootstrap required, opened a new log
I20260809 15:10:35.206537 16657 ts_tablet_manager.cc:1397] T 837cbee498114752a8f0f2b7a1557570 P 4e57e56718f64d37a9ee564ebad08a4e: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.002s
I20260809 15:10:35.207464 16660 tablet_bootstrap.cc:654] T 837cbee498114752a8f0f2b7a1557570 P 2949328774234d318318d0202b04b980: Neither blocks nor log segments found. Creating new log.
I20260809 15:10:35.208230 16661 tablet_bootstrap.cc:654] T 837cbee498114752a8f0f2b7a1557570 P 0b26588483d746bdba86f18f39b0ca50: Neither blocks nor log segments found. Creating new log.
I20260809 15:10:35.208743 16660 log.cc:824] T 837cbee498114752a8f0f2b7a1557570 P 2949328774234d318318d0202b04b980: Log is configured to *not* fsync() on all Append() calls
I20260809 15:10:35.209465 16661 log.cc:824] T 837cbee498114752a8f0f2b7a1557570 P 0b26588483d746bdba86f18f39b0ca50: Log is configured to *not* fsync() on all Append() calls
I20260809 15:10:35.211797 16660 tablet_bootstrap.cc:492] T 837cbee498114752a8f0f2b7a1557570 P 2949328774234d318318d0202b04b980: No bootstrap required, opened a new log
I20260809 15:10:35.212110 16660 ts_tablet_manager.cc:1397] T 837cbee498114752a8f0f2b7a1557570 P 2949328774234d318318d0202b04b980: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260809 15:10:35.212239 16661 tablet_bootstrap.cc:492] T 837cbee498114752a8f0f2b7a1557570 P 0b26588483d746bdba86f18f39b0ca50: No bootstrap required, opened a new log
I20260809 15:10:35.212823 16661 ts_tablet_manager.cc:1397] T 837cbee498114752a8f0f2b7a1557570 P 0b26588483d746bdba86f18f39b0ca50: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260809 15:10:35.215760 16657 raft_consensus.cc:348] T 837cbee498114752a8f0f2b7a1557570 P 4e57e56718f64d37a9ee564ebad08a4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b26588483d746bdba86f18f39b0ca50" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46385 } } peers { permanent_uuid: "4e57e56718f64d37a9ee564ebad08a4e" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 37317 } } peers { permanent_uuid: "2949328774234d318318d0202b04b980" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36181 } }
I20260809 15:10:35.216257 16657 raft_consensus.cc:374] T 837cbee498114752a8f0f2b7a1557570 P 4e57e56718f64d37a9ee564ebad08a4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:10:35.216377 16657 raft_consensus.cc:729] T 837cbee498114752a8f0f2b7a1557570 P 4e57e56718f64d37a9ee564ebad08a4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e57e56718f64d37a9ee564ebad08a4e, State: Initialized, Role: FOLLOWER
I20260809 15:10:35.216984 16657 consensus_queue.cc:260] T 837cbee498114752a8f0f2b7a1557570 P 4e57e56718f64d37a9ee564ebad08a4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b26588483d746bdba86f18f39b0ca50" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46385 } } peers { permanent_uuid: "4e57e56718f64d37a9ee564ebad08a4e" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 37317 } } peers { permanent_uuid: "2949328774234d318318d0202b04b980" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36181 } }
I20260809 15:10:35.218086 16184 heartbeater.cc:503] Master 127.24.183.254:41811 was elected leader, sending a full tablet report...
I20260809 15:10:35.218667 16657 ts_tablet_manager.cc:1428] T 837cbee498114752a8f0f2b7a1557570 P 4e57e56718f64d37a9ee564ebad08a4e: Time spent starting tablet: real 0.012s user 0.010s sys 0.002s
I20260809 15:10:35.221261 16660 raft_consensus.cc:348] T 837cbee498114752a8f0f2b7a1557570 P 2949328774234d318318d0202b04b980 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b26588483d746bdba86f18f39b0ca50" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46385 } } peers { permanent_uuid: "4e57e56718f64d37a9ee564ebad08a4e" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 37317 } } peers { permanent_uuid: "2949328774234d318318d0202b04b980" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36181 } }
I20260809 15:10:35.221586 16660 raft_consensus.cc:374] T 837cbee498114752a8f0f2b7a1557570 P 2949328774234d318318d0202b04b980 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:10:35.221727 16660 raft_consensus.cc:729] T 837cbee498114752a8f0f2b7a1557570 P 2949328774234d318318d0202b04b980 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2949328774234d318318d0202b04b980, State: Initialized, Role: FOLLOWER
I20260809 15:10:35.222177 16661 raft_consensus.cc:348] T 837cbee498114752a8f0f2b7a1557570 P 0b26588483d746bdba86f18f39b0ca50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b26588483d746bdba86f18f39b0ca50" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46385 } } peers { permanent_uuid: "4e57e56718f64d37a9ee564ebad08a4e" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 37317 } } peers { permanent_uuid: "2949328774234d318318d0202b04b980" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36181 } }
I20260809 15:10:35.222484 16661 raft_consensus.cc:374] T 837cbee498114752a8f0f2b7a1557570 P 0b26588483d746bdba86f18f39b0ca50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:10:35.222303 16660 consensus_queue.cc:260] T 837cbee498114752a8f0f2b7a1557570 P 2949328774234d318318d0202b04b980 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b26588483d746bdba86f18f39b0ca50" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46385 } } peers { permanent_uuid: "4e57e56718f64d37a9ee564ebad08a4e" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 37317 } } peers { permanent_uuid: "2949328774234d318318d0202b04b980" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36181 } }
I20260809 15:10:35.222581 16661 raft_consensus.cc:729] T 837cbee498114752a8f0f2b7a1557570 P 0b26588483d746bdba86f18f39b0ca50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b26588483d746bdba86f18f39b0ca50, State: Initialized, Role: FOLLOWER
I20260809 15:10:35.223194 16661 consensus_queue.cc:260] T 837cbee498114752a8f0f2b7a1557570 P 0b26588483d746bdba86f18f39b0ca50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b26588483d746bdba86f18f39b0ca50" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46385 } } peers { permanent_uuid: "4e57e56718f64d37a9ee564ebad08a4e" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 37317 } } peers { permanent_uuid: "2949328774234d318318d0202b04b980" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36181 } }
I20260809 15:10:35.223457 16631 heartbeater.cc:503] Master 127.24.183.254:41811 was elected leader, sending a full tablet report...
I20260809 15:10:35.224192 16660 ts_tablet_manager.cc:1428] T 837cbee498114752a8f0f2b7a1557570 P 2949328774234d318318d0202b04b980: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260809 15:10:35.225337 16408 heartbeater.cc:503] Master 127.24.183.254:41811 was elected leader, sending a full tablet report...
I20260809 15:10:35.226481 16661 ts_tablet_manager.cc:1428] T 837cbee498114752a8f0f2b7a1557570 P 0b26588483d746bdba86f18f39b0ca50: Time spent starting tablet: real 0.013s user 0.009s sys 0.002s
W20260809 15:10:35.347543 16632 tablet.cc:2369] T 837cbee498114752a8f0f2b7a1557570 P 2949328774234d318318d0202b04b980: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:10:35.382036 16409 tablet.cc:2369] T 837cbee498114752a8f0f2b7a1557570 P 0b26588483d746bdba86f18f39b0ca50: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:10:35.410022 16671 raft_consensus.cc:482] T 837cbee498114752a8f0f2b7a1557570 P 0b26588483d746bdba86f18f39b0ca50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:10:35.410331 16671 raft_consensus.cc:504] T 837cbee498114752a8f0f2b7a1557570 P 0b26588483d746bdba86f18f39b0ca50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b26588483d746bdba86f18f39b0ca50" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46385 } } peers { permanent_uuid: "4e57e56718f64d37a9ee564ebad08a4e" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 37317 } } peers { permanent_uuid: "2949328774234d318318d0202b04b980" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36181 } }
I20260809 15:10:35.412102 16671 leader_election.cc:288] T 837cbee498114752a8f0f2b7a1557570 P 0b26588483d746bdba86f18f39b0ca50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e57e56718f64d37a9ee564ebad08a4e (127.24.183.193:37317), 2949328774234d318318d0202b04b980 (127.24.183.195:36181)
I20260809 15:10:35.419158 16656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:35.412698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:49943 (local address 127.24.183.195:36181)
0809 15:10:35.412988 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:35.412992 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:35.413961 (+ 969us) server_negotiation.cc:407] Connection header received
0809 15:10:35.414129 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:35.414133 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:35.414194 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:35.414269 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:10:35.414881 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:35.415582 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:35.417414 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:35.418206 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:35.418352 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:35.418430 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:35.418758 (+ 328us) server_negotiation.cc:299] Negotiation successful
0809 15:10:35.418899 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154}
I20260809 15:10:35.419165 16654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:35.412660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:60695 (local address 127.24.183.193:37317)
0809 15:10:35.412883 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:35.412890 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:35.413881 (+ 991us) server_negotiation.cc:407] Connection header received
0809 15:10:35.414027 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:35.414033 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:35.414101 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:35.414176 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:10:35.414721 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:35.415439 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:35.417443 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:35.418231 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:35.418370 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:35.418441 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:35.418776 (+ 335us) server_negotiation.cc:299] Negotiation successful
0809 15:10:35.418912 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 15:10:35.419317 16675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:35.412557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:36181 (local address 127.24.183.194:49943)
0809 15:10:35.413866 (+ 1309us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:35.413881 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:35.413998 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:35.414305 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:35.414309 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:35.414347 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:10:35.414733 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:35.414742 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:35.415735 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:35.415740 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:35.417260 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:35.417272 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:35.418431 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:35.418441 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:35.418609 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:35.418636 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 15:10:35.418717 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 15:10:35.418839 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1191,"mutex_wait_us":86,"thread_start_us":819,"threads_started":1}
I20260809 15:10:35.419564 16674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:35.412556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:37317 (local address 127.24.183.194:60695)
0809 15:10:35.413733 (+ 1177us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:35.413749 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:35.413898 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:35.414200 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:35.414204 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:35.414226 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:10:35.414569 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:35.414581 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:35.415562 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:35.415567 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:35.417276 (+ 1709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:35.417288 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:35.418458 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:35.418464 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:35.418635 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:35.418658 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 15:10:35.418735 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 15:10:35.418852 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1031,"mutex_wait_us":67,"thread_start_us":640,"threads_started":1}
I20260809 15:10:35.420361 16560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "837cbee498114752a8f0f2b7a1557570" candidate_uuid: "0b26588483d746bdba86f18f39b0ca50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2949328774234d318318d0202b04b980" is_pre_election: true
I20260809 15:10:35.420437 16103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "837cbee498114752a8f0f2b7a1557570" candidate_uuid: "0b26588483d746bdba86f18f39b0ca50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e57e56718f64d37a9ee564ebad08a4e" is_pre_election: true
I20260809 15:10:35.420842 16560 raft_consensus.cc:2454] T 837cbee498114752a8f0f2b7a1557570 P 2949328774234d318318d0202b04b980 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b26588483d746bdba86f18f39b0ca50 in term 0.
I20260809 15:10:35.420900 16103 raft_consensus.cc:2454] T 837cbee498114752a8f0f2b7a1557570 P 4e57e56718f64d37a9ee564ebad08a4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b26588483d746bdba86f18f39b0ca50 in term 0.
I20260809 15:10:35.421514 16212 leader_election.cc:302] T 837cbee498114752a8f0f2b7a1557570 P 0b26588483d746bdba86f18f39b0ca50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b26588483d746bdba86f18f39b0ca50, 4e57e56718f64d37a9ee564ebad08a4e; no voters:
I20260809 15:10:35.421916 16671 raft_consensus.cc:2781] T 837cbee498114752a8f0f2b7a1557570 P 0b26588483d746bdba86f18f39b0ca50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:10:35.422044 16671 raft_consensus.cc:482] T 837cbee498114752a8f0f2b7a1557570 P 0b26588483d746bdba86f18f39b0ca50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:10:35.422137 16671 raft_consensus.cc:3037] T 837cbee498114752a8f0f2b7a1557570 P 0b26588483d746bdba86f18f39b0ca50 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:10:35.424537 16671 raft_consensus.cc:504] T 837cbee498114752a8f0f2b7a1557570 P 0b26588483d746bdba86f18f39b0ca50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b26588483d746bdba86f18f39b0ca50" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46385 } } peers { permanent_uuid: "4e57e56718f64d37a9ee564ebad08a4e" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 37317 } } peers { permanent_uuid: "2949328774234d318318d0202b04b980" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36181 } }
I20260809 15:10:35.425091 16671 leader_election.cc:288] T 837cbee498114752a8f0f2b7a1557570 P 0b26588483d746bdba86f18f39b0ca50 [CANDIDATE]: Term 1 election: Requested vote from peers 4e57e56718f64d37a9ee564ebad08a4e (127.24.183.193:37317), 2949328774234d318318d0202b04b980 (127.24.183.195:36181)
I20260809 15:10:35.425494 16103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "837cbee498114752a8f0f2b7a1557570" candidate_uuid: "0b26588483d746bdba86f18f39b0ca50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e57e56718f64d37a9ee564ebad08a4e"
I20260809 15:10:35.425508 16560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "837cbee498114752a8f0f2b7a1557570" candidate_uuid: "0b26588483d746bdba86f18f39b0ca50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2949328774234d318318d0202b04b980"
I20260809 15:10:35.425663 16103 raft_consensus.cc:3037] T 837cbee498114752a8f0f2b7a1557570 P 4e57e56718f64d37a9ee564ebad08a4e [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:10:35.425657 16560 raft_consensus.cc:3037] T 837cbee498114752a8f0f2b7a1557570 P 2949328774234d318318d0202b04b980 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:10:35.427852 16560 raft_consensus.cc:2454] T 837cbee498114752a8f0f2b7a1557570 P 2949328774234d318318d0202b04b980 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b26588483d746bdba86f18f39b0ca50 in term 1.
I20260809 15:10:35.427855 16103 raft_consensus.cc:2454] T 837cbee498114752a8f0f2b7a1557570 P 4e57e56718f64d37a9ee564ebad08a4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b26588483d746bdba86f18f39b0ca50 in term 1.
I20260809 15:10:35.428218 16212 leader_election.cc:302] T 837cbee498114752a8f0f2b7a1557570 P 0b26588483d746bdba86f18f39b0ca50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0b26588483d746bdba86f18f39b0ca50, 2949328774234d318318d0202b04b980, 4e57e56718f64d37a9ee564ebad08a4e; no voters:
I20260809 15:10:35.428510 16671 raft_consensus.cc:2781] T 837cbee498114752a8f0f2b7a1557570 P 0b26588483d746bdba86f18f39b0ca50 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:10:35.428900 16671 raft_consensus.cc:686] T 837cbee498114752a8f0f2b7a1557570 P 0b26588483d746bdba86f18f39b0ca50 [term 1 LEADER]: Becoming Leader. State: Replica: 0b26588483d746bdba86f18f39b0ca50, State: Running, Role: LEADER
I20260809 15:10:35.429327 16671 consensus_queue.cc:237] T 837cbee498114752a8f0f2b7a1557570 P 0b26588483d746bdba86f18f39b0ca50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b26588483d746bdba86f18f39b0ca50" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46385 } } peers { permanent_uuid: "4e57e56718f64d37a9ee564ebad08a4e" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 37317 } } peers { permanent_uuid: "2949328774234d318318d0202b04b980" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36181 } }
I20260809 15:10:35.433833 15850 catalog_manager.cc:5495] T 837cbee498114752a8f0f2b7a1557570 P 0b26588483d746bdba86f18f39b0ca50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b26588483d746bdba86f18f39b0ca50 (127.24.183.194). New cstate: current_term: 1 leader_uuid: "0b26588483d746bdba86f18f39b0ca50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b26588483d746bdba86f18f39b0ca50" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 46385 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e57e56718f64d37a9ee564ebad08a4e" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 37317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2949328774234d318318d0202b04b980" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36181 } health_report { overall_health: UNKNOWN } } }
W20260809 15:10:35.435084 16185 tablet.cc:2369] T 837cbee498114752a8f0f2b7a1557570 P 4e57e56718f64d37a9ee564ebad08a4e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:10:35.466243 16652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:35.460881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50886 (local address 127.24.183.194:46385)
0809 15:10:35.461068 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:35.461073 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:35.461091 (+ 18us) server_negotiation.cc:407] Connection header received
0809 15:10:35.461342 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:35.461346 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:35.461424 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:35.461538 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:10:35.461988 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:35.462676 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:35.464277 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:35.464677 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:35.464767 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:35.465233 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:10:35.465688 (+ 455us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:10:35.465817 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:35.465945 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 15:10:35.466020 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 15:10:35.474145 16560 raft_consensus.cc:1261] T 837cbee498114752a8f0f2b7a1557570 P 2949328774234d318318d0202b04b980 [term 1 FOLLOWER]: Refusing update from remote peer 0b26588483d746bdba86f18f39b0ca50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:10:35.474159 16103 raft_consensus.cc:1261] T 837cbee498114752a8f0f2b7a1557570 P 4e57e56718f64d37a9ee564ebad08a4e [term 1 FOLLOWER]: Refusing update from remote peer 0b26588483d746bdba86f18f39b0ca50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:10:35.475019 16679 consensus_queue.cc:1035] T 837cbee498114752a8f0f2b7a1557570 P 0b26588483d746bdba86f18f39b0ca50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2949328774234d318318d0202b04b980" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 36181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:10:35.475333 16671 consensus_queue.cc:1035] T 837cbee498114752a8f0f2b7a1557570 P 0b26588483d746bdba86f18f39b0ca50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e57e56718f64d37a9ee564ebad08a4e" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 37317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:10:35.486976 16689 mvcc.cc:205] Tried to move back new op lower bound from 7316636612492505088 to 7316636612321894400. Current Snapshot: MvccSnapshot[applied={T|T < 7316636612492505088}]
I20260809 15:10:35.499079 15849 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:34624:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:10:35.499398 15849 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:34624:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:10:35.503507 15849 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951: Sending DeleteTablet for 3 replicas of tablet 837cbee498114752a8f0f2b7a1557570
I20260809 15:10:35.504633 16294 tablet_service.cc:1514] Processing DeleteTablet for tablet 837cbee498114752a8f0f2b7a1557570 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:10:35 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:50882
I20260809 15:10:35.504647 16524 tablet_service.cc:1514] Processing DeleteTablet for tablet 837cbee498114752a8f0f2b7a1557570 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:10:35 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:56988
I20260809 15:10:35.504875 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 15923
I20260809 15:10:35.505309 16065 tablet_service.cc:1514] Processing DeleteTablet for tablet 837cbee498114752a8f0f2b7a1557570 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:10:35 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:47772
I20260809 15:10:35.506318 16699 tablet_replica.cc:330] T 837cbee498114752a8f0f2b7a1557570 P 0b26588483d746bdba86f18f39b0ca50: stopping tablet replica
I20260809 15:10:35.508597 16699 raft_consensus.cc:2229] T 837cbee498114752a8f0f2b7a1557570 P 0b26588483d746bdba86f18f39b0ca50 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:10:35.509224 16699 raft_consensus.cc:2258] T 837cbee498114752a8f0f2b7a1557570 P 0b26588483d746bdba86f18f39b0ca50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:10:35.509719 16701 tablet_replica.cc:330] T 837cbee498114752a8f0f2b7a1557570 P 2949328774234d318318d0202b04b980: stopping tablet replica
I20260809 15:10:35.510229 16701 raft_consensus.cc:2229] T 837cbee498114752a8f0f2b7a1557570 P 2949328774234d318318d0202b04b980 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:10:35.510778 16701 raft_consensus.cc:2258] T 837cbee498114752a8f0f2b7a1557570 P 2949328774234d318318d0202b04b980 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:10:35.514554 16699 ts_tablet_manager.cc:1905] T 837cbee498114752a8f0f2b7a1557570 P 0b26588483d746bdba86f18f39b0ca50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:10:35.518925 16702 tablet_replica.cc:330] T 837cbee498114752a8f0f2b7a1557570 P 4e57e56718f64d37a9ee564ebad08a4e: stopping tablet replica
I20260809 15:10:35.519430 16702 raft_consensus.cc:2229] T 837cbee498114752a8f0f2b7a1557570 P 4e57e56718f64d37a9ee564ebad08a4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:10:35.519925 16702 raft_consensus.cc:2258] T 837cbee498114752a8f0f2b7a1557570 P 4e57e56718f64d37a9ee564ebad08a4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:10:35.521941 16702 ts_tablet_manager.cc:1905] T 837cbee498114752a8f0f2b7a1557570 P 4e57e56718f64d37a9ee564ebad08a4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:10:35.522187 16699 ts_tablet_manager.cc:1918] T 837cbee498114752a8f0f2b7a1557570 P 0b26588483d746bdba86f18f39b0ca50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:10:35.522342 16699 log.cc:1196] T 837cbee498114752a8f0f2b7a1557570 P 0b26588483d746bdba86f18f39b0ca50: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/wal/wals/837cbee498114752a8f0f2b7a1557570
I20260809 15:10:35.522946 16699 ts_tablet_manager.cc:1939] T 837cbee498114752a8f0f2b7a1557570 P 0b26588483d746bdba86f18f39b0ca50: Deleting consensus metadata
I20260809 15:10:35.524926 15829 catalog_manager.cc:4841] TS 0b26588483d746bdba86f18f39b0ca50 (127.24.183.194:46385): tablet 837cbee498114752a8f0f2b7a1557570 (table test-table [id=7554ecbce8d6404b8a85c91b837f7a71]) successfully deleted
I20260809 15:10:35.525799 16701 ts_tablet_manager.cc:1905] T 837cbee498114752a8f0f2b7a1557570 P 2949328774234d318318d0202b04b980: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:10:35.527380 16654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:35.505679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47784 (local address 127.24.183.193:37317)
0809 15:10:35.505868 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:35.505875 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:35.506030 (+ 155us) server_negotiation.cc:407] Connection header received
0809 15:10:35.506201 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:35.506207 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:35.506285 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:35.506402 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:35.508213 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:35.509489 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:35.511432 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:35.512218 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:35.512348 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:35.519807 (+ 7459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:35.519820 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:35.519836 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:35.519879 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:35.523704 (+ 3825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:35.524797 (+ 1093us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:35.524807 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:35.524811 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:35.524972 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:35.525468 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:35.525474 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:35.525478 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:35.526036 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:35.526351 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:35.526690 (+ 339us) server_negotiation.cc:299] Negotiation successful
0809 15:10:35.526966 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 15:10:35.528097 16169 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:10:35.534806 16701 ts_tablet_manager.cc:1918] T 837cbee498114752a8f0f2b7a1557570 P 2949328774234d318318d0202b04b980: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:10:35.534973 16701 log.cc:1196] T 837cbee498114752a8f0f2b7a1557570 P 2949328774234d318318d0202b04b980: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/wal/wals/837cbee498114752a8f0f2b7a1557570
I20260809 15:10:35.535553 16701 ts_tablet_manager.cc:1939] T 837cbee498114752a8f0f2b7a1557570 P 2949328774234d318318d0202b04b980: Deleting consensus metadata
I20260809 15:10:35.537418 15829 catalog_manager.cc:4841] TS 2949328774234d318318d0202b04b980 (127.24.183.195:36181): tablet 837cbee498114752a8f0f2b7a1557570 (table test-table [id=7554ecbce8d6404b8a85c91b837f7a71]) successfully deleted
I20260809 15:10:35.620198 16702 ts_tablet_manager.cc:1918] T 837cbee498114752a8f0f2b7a1557570 P 4e57e56718f64d37a9ee564ebad08a4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:10:35.620432 16702 log.cc:1196] T 837cbee498114752a8f0f2b7a1557570 P 4e57e56718f64d37a9ee564ebad08a4e: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/wal/wals/837cbee498114752a8f0f2b7a1557570
I20260809 15:10:35.620988 16702 ts_tablet_manager.cc:1939] T 837cbee498114752a8f0f2b7a1557570 P 4e57e56718f64d37a9ee564ebad08a4e: Deleting consensus metadata
I20260809 15:10:35.621088 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 15923
I20260809 15:10:35.622682 15823 catalog_manager.cc:4841] TS 4e57e56718f64d37a9ee564ebad08a4e (127.24.183.193:37317): tablet 837cbee498114752a8f0f2b7a1557570 (table test-table [id=7554ecbce8d6404b8a85c91b837f7a71]) successfully deleted
I20260809 15:10:35.661013 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 16189
I20260809 15:10:35.680601 16652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:35.661746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50898 (local address 127.24.183.194:46385)
0809 15:10:35.661941 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:35.661947 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:35.662105 (+ 158us) server_negotiation.cc:407] Connection header received
0809 15:10:35.662290 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:35.662294 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:35.662365 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:35.662447 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:35.663765 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:35.664602 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:35.665950 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:35.666380 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:35.666497 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:35.669065 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:35.669076 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:35.669090 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:35.669126 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:35.672671 (+ 3545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:35.673465 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:35.673474 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:35.673478 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:35.673649 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:35.674034 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:35.674039 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:35.674042 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:35.674520 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:35.674836 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:35.680041 (+ 5205us) server_negotiation.cc:299] Negotiation successful
0809 15:10:35.680254 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 15:10:35.681510 16401 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:10:35.797716 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 16189
I20260809 15:10:35.823539 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 16411
I20260809 15:10:35.836071 16656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:35.824304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56994 (local address 127.24.183.195:36181)
0809 15:10:35.824501 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:35.824505 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:35.824542 (+ 37us) server_negotiation.cc:407] Connection header received
0809 15:10:35.824713 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:35.824717 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:35.824791 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:35.824868 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:35.826095 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:35.826826 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:35.828189 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:35.828576 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:35.828667 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:35.831080 (+ 2413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:35.831093 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:35.831111 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:35.831145 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:35.833742 (+ 2597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:35.834337 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:35.834342 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:35.834345 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:35.834488 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:35.834745 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:35.834747 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:35.834749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:35.835085 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:35.835357 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:35.835540 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 15:10:35.835765 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 15:10:35.836752 16624 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:10:35.927016 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 16411
I20260809 15:10:35.949450 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 15788
I20260809 15:10:35.950555 15908 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:10:36.066671 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 15788
I20260809 15:10:36.098183 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:41811
--webserver_interface=127.24.183.254
--webserver_port=41829
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:33207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:41811
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:10:36.433056 16712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:10:36.433336 16712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:10:36.433470 16712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:10:36.440555 16712 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:10:36.440667 16712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:10:36.440744 16712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:10:36.440830 16712 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:10:36.440887 16712 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:10:36.450390 16712 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:33207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:41811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:41811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=41829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:10:36.451604 16712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:10:36.452898 16712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:10:36.464771 16718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:36.464823 16720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:36.464737 16717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:10:36.465555 16712 server_base.cc:1034] running on GCE node
I20260809 15:10:36.466253 16712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:10:36.467510 16712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:10:36.468711 16712 hybrid_clock.cc:648] HybridClock initialized: now 1786288236468653 us; error 42 us; skew 500 ppm
I20260809 15:10:36.469010 16712 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:10:36 dist-test-slave-nj21 krb5kdc[14825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288236, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:10:36.472060 16712 init.cc:377] Logged in from keytab as kudu/127.24.183.254@KRBTEST.COM (short username kudu)
I20260809 15:10:36.473505 16712 webserver.cc:460] Webserver started at http://127.24.183.254:41829/ using document root <none> and password file <none>
I20260809 15:10:36.474116 16712 fs_manager.cc:359] Metadata directory not provided
I20260809 15:10:36.474218 16712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:10:36.479318 16712 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:10:36.481971 16731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:36.483187 16712 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 15:10:36.483318 16712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "49997ba886fa4ba9b145e19999ab8951"
format_stamp: "Formatted at 2026-08-09 15:10:30 on dist-test-slave-nj21"
server_key: "90039f93bb59fa1a23ff0e4d8ba72425"
server_key_iv: "b1505f0f0c2c6d93cd9902ca659cdeb1"
server_key_version: "encryptionkey@0"
I20260809 15:10:36.484304 16712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:10:36.504175 16712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:10:36.507820 16712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:10:36.508030 16712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:10:36.525730 16712 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:41811
I20260809 15:10:36.525740 16829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:41811 every 8 connection(s)
I20260809 15:10:36.527185 16712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:10:36.533373 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 16712
I20260809 15:10:36.534317 16830 sys_catalog.cc:263] Verifying existing consensus state
I20260809 15:10:36.534405 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.193:37317
--local_ip_for_outbound_sockets=127.24.183.193
--tserver_master_addrs=127.24.183.254:41811
--webserver_port=35243
--webserver_interface=127.24.183.193
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:33207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:10:36.538693 16830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951: Bootstrap starting.
I20260809 15:10:36.561028 16830 log.cc:824] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951: Log is configured to *not* fsync() on all Append() calls
I20260809 15:10:36.577418 16830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:10:36.577925 16830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951: Bootstrap complete.
I20260809 15:10:36.586885 16830 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49997ba886fa4ba9b145e19999ab8951" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 41811 } }
I20260809 15:10:36.587899 16830 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49997ba886fa4ba9b145e19999ab8951, State: Initialized, Role: FOLLOWER
I20260809 15:10:36.588511 16830 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49997ba886fa4ba9b145e19999ab8951" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 41811 } }
I20260809 15:10:36.588773 16830 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:10:36.588886 16830 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:10:36.589051 16830 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:10:36.592851 16830 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49997ba886fa4ba9b145e19999ab8951" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 41811 } }
I20260809 15:10:36.593322 16830 leader_election.cc:302] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49997ba886fa4ba9b145e19999ab8951; no voters:
I20260809 15:10:36.593958 16830 leader_election.cc:288] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 15:10:36.594096 16836 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:10:36.595162 16836 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [term 3 LEADER]: Becoming Leader. State: Replica: 49997ba886fa4ba9b145e19999ab8951, State: Running, Role: LEADER
I20260809 15:10:36.595813 16836 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49997ba886fa4ba9b145e19999ab8951" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 41811 } }
I20260809 15:10:36.596117 16830 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:10:36.598269 16839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49997ba886fa4ba9b145e19999ab8951. Latest consensus state: current_term: 3 leader_uuid: "49997ba886fa4ba9b145e19999ab8951" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49997ba886fa4ba9b145e19999ab8951" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 41811 } } }
I20260809 15:10:36.598639 16839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [sys.catalog]: This master's current role is: LEADER
I20260809 15:10:36.599174 16837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "49997ba886fa4ba9b145e19999ab8951" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49997ba886fa4ba9b145e19999ab8951" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 41811 } } }
I20260809 15:10:36.599435 16837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49997ba886fa4ba9b145e19999ab8951 [sys.catalog]: This master's current role is: LEADER
I20260809 15:10:36.608228 16847 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:10:36.618913 16847 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:10:36.620440 16847 catalog_manager.cc:1259] Loaded cluster ID: 67744b2672344797b8ec69afdbdf09a1
I20260809 15:10:36.620607 16847 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 15:10:36.622766 16847 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fa3561549ad google::LogMessage::Fail() at ??:0
@ 0x7fa35615788a google::LogMessage::SendToLog() at ??:0
@ 0x7fa3561544de google::LogMessage::Flush() at ??:0
@ 0x7fa356155a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fa36bb87512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fa36bb84976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fa36bc0c610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fa36bc0c439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fa358da0345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fa358da391c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fa358da3799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fa358d7eeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fa35c0586db start_thread at ??:0
@ 0x7fa354fee61f clone at ??:0
W20260809 15:10:36.904271 16834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:10:36.904551 16834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:10:36.904681 16834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:10:36.911859 16834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:10:36.911971 16834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:10:36.912138 16834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.193
I20260809 15:10:36.921614 16834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:33207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.193:37317
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.183.193
--webserver_port=35243
--tserver_master_addrs=127.24.183.254:41811
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.193
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:10:36.922863 16834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:10:36.924173 16834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:10:36.936762 16894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:36.936833 16897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:36.936822 16895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:10:36.937693 16834 server_base.cc:1034] running on GCE node
I20260809 15:10:36.938252 16834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:10:36.939105 16834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:10:36.940284 16834 hybrid_clock.cc:648] HybridClock initialized: now 1786288236940219 us; error 53 us; skew 500 ppm
I20260809 15:10:36.940600 16834 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:10:36 dist-test-slave-nj21 krb5kdc[14825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288236, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:10:36.943616 16834 init.cc:377] Logged in from keytab as kudu/127.24.183.193@KRBTEST.COM (short username kudu)
I20260809 15:10:36.944974 16834 webserver.cc:460] Webserver started at http://127.24.183.193:35243/ using document root <none> and password file <none>
I20260809 15:10:36.945592 16834 fs_manager.cc:359] Metadata directory not provided
I20260809 15:10:36.945710 16834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:10:36.950759 16834 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.001s
I20260809 15:10:36.953393 16905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:36.954617 16834 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260809 15:10:36.954771 16834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/wal
uuid: "4e57e56718f64d37a9ee564ebad08a4e"
format_stamp: "Formatted at 2026-08-09 15:10:31 on dist-test-slave-nj21"
server_key: "cf13abac65bace41da3d311e65d63473"
server_key_iv: "74bb9cf8d79954e751ec3d9cb04aab06"
server_key_version: "encryptionkey@0"
I20260809 15:10:36.955770 16834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:10:36.973471 16834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:10:36.976960 16834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:10:36.977205 16834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:10:36.978125 16834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:10:36.979398 16834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:10:36.979491 16834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:36.979601 16834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:10:36.979693 16834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:37.011724 16834 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.193:37317
I20260809 15:10:37.011771 17103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.193:37317 every 8 connection(s)
I20260809 15:10:37.013216 16834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
I20260809 15:10:37.018082 17110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:37.016183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.193:46835)
0809 15:10:37.016915 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:37.017130 (+ 215us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":273,"threads_started":1}
W20260809 15:10:37.018693 17104 heartbeater.cc:644] Failed to heartbeat to 127.24.183.254:41811 (0 consecutive failures): Network error: Failed to ping master at 127.24.183.254:41811: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
I20260809 15:10:37.019475 17110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:37.019165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.193:49627)
0809 15:10:37.019300 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:37.019375 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 15:10:37.020341 17110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:37.020004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.193:33107)
0809 15:10:37.020139 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:37.020214 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 15:10:37.020565 17104 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 15:10:37.023494 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 16834
I20260809 15:10:37.024159 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.194:46385
--local_ip_for_outbound_sockets=127.24.183.194
--tserver_master_addrs=127.24.183.254:41811
--webserver_port=33949
--webserver_interface=127.24.183.194
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:33207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:10:37.359722 17111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:10:37.360021 17111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:10:37.360173 17111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:10:37.367517 17111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:10:37.367622 17111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:10:37.367806 17111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.194
I20260809 15:10:37.377553 17111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:33207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.194:46385
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.183.194
--webserver_port=33949
--tserver_master_addrs=127.24.183.254:41811
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.194
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:10:37.378770 17111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:10:37.380136 17111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:10:37.392665 17121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:37.392690 17122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:37.392841 17124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:10:37.393510 17111 server_base.cc:1034] running on GCE node
I20260809 15:10:37.394069 17111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:10:37.394923 17111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:10:37.396118 17111 hybrid_clock.cc:648] HybridClock initialized: now 1786288237396077 us; error 28 us; skew 500 ppm
I20260809 15:10:37.396412 17111 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:10:37 dist-test-slave-nj21 krb5kdc[14825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288237, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:10:37.399399 17111 init.cc:377] Logged in from keytab as kudu/127.24.183.194@KRBTEST.COM (short username kudu)
I20260809 15:10:37.400832 17111 webserver.cc:460] Webserver started at http://127.24.183.194:33949/ using document root <none> and password file <none>
I20260809 15:10:37.401489 17111 fs_manager.cc:359] Metadata directory not provided
I20260809 15:10:37.401610 17111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:10:37.406694 17111 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 15:10:37.409303 17136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:37.410550 17111 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 15:10:37.410694 17111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/wal
uuid: "0b26588483d746bdba86f18f39b0ca50"
format_stamp: "Formatted at 2026-08-09 15:10:31 on dist-test-slave-nj21"
server_key: "0daae33d260d9132d5fb962faa6f6c75"
server_key_iv: "98aa459986d440a9d2a7d13bfa7fc021"
server_key_version: "encryptionkey@0"
I20260809 15:10:37.411631 17111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:10:37.437412 17111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:10:37.441295 17111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:10:37.441540 17111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:10:37.442489 17111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:10:37.443861 17111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:10:37.443988 17111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:37.444109 17111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:10:37.444183 17111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:37.475001 17111 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.194:46385
I20260809 15:10:37.475066 17315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.194:46385 every 8 connection(s)
I20260809 15:10:37.476506 17111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
I20260809 15:10:37.479892 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 17111
I20260809 15:10:37.480691 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.195:36181
--local_ip_for_outbound_sockets=127.24.183.195
--tserver_master_addrs=127.24.183.254:41811
--webserver_port=46585
--webserver_interface=127.24.183.195
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:33207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:10:37.481647 17318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:37.479355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.194:50693)
0809 15:10:37.480197 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:37.480498 (+ 301us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":322,"threads_started":1}
W20260809 15:10:37.482311 17316 heartbeater.cc:644] Failed to heartbeat to 127.24.183.254:41811 (0 consecutive failures): Network error: Failed to ping master at 127.24.183.254:41811: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
I20260809 15:10:37.483132 17318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:37.482763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.194:38179)
0809 15:10:37.482924 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:37.483003 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 15:10:37.483992 17318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:37.483621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.194:56377)
0809 15:10:37.483773 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:37.483867 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260809 15:10:37.484262 17316 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 15:10:37.810737 17320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:10:37.810994 17320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:10:37.811095 17320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:10:37.818135 17320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:10:37.818228 17320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:10:37.818343 17320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.195
I20260809 15:10:37.827653 17320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:33207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.195:36181
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.183.195
--webserver_port=46585
--tserver_master_addrs=127.24.183.254:41811
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.195
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:10:37.828860 17320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:10:37.830111 17320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:10:37.842706 17330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:37.842661 17335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:37.842746 17331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:10:37.843520 17320 server_base.cc:1034] running on GCE node
I20260809 15:10:37.844125 17320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:10:37.844973 17320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:10:37.846155 17320 hybrid_clock.cc:648] HybridClock initialized: now 1786288237846103 us; error 43 us; skew 500 ppm
I20260809 15:10:37.846431 17320 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:10:37 dist-test-slave-nj21 krb5kdc[14825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288237, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:10:37.849469 17320 init.cc:377] Logged in from keytab as kudu/127.24.183.195@KRBTEST.COM (short username kudu)
I20260809 15:10:37.850831 17320 webserver.cc:460] Webserver started at http://127.24.183.195:46585/ using document root <none> and password file <none>
I20260809 15:10:37.851464 17320 fs_manager.cc:359] Metadata directory not provided
I20260809 15:10:37.851579 17320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:10:37.856662 17320 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:10:37.859179 17344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:37.860523 17320 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:10:37.860692 17320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/wal
uuid: "2949328774234d318318d0202b04b980"
format_stamp: "Formatted at 2026-08-09 15:10:32 on dist-test-slave-nj21"
server_key: "ed8b8d2ec7489bf0925cadc31dd75000"
server_key_iv: "e1468b76f8d7e3bab1ad91c33934fb32"
server_key_version: "encryptionkey@0"
I20260809 15:10:37.861812 17320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:10:37.884146 17320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:10:37.887876 17320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:10:37.888129 17320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:10:37.889082 17320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:10:37.890414 17320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:10:37.890523 17320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:37.890631 17320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:10:37.890702 17320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:37.921590 17320 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.195:36181
I20260809 15:10:37.921657 17535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.195:36181 every 8 connection(s)
I20260809 15:10:37.923025 17320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
I20260809 15:10:37.927325 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 17320
I20260809 15:10:37.927886 17538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:37.925982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.195:45383)
0809 15:10:37.926711 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:37.926970 (+ 259us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":278,"threads_started":1}
W20260809 15:10:37.928561 17536 heartbeater.cc:644] Failed to heartbeat to 127.24.183.254:41811 (0 consecutive failures): Network error: Failed to ping master at 127.24.183.254:41811: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
I20260809 15:10:37.929358 17538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:37.929023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.195:58919)
0809 15:10:37.929166 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:37.929235 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 15:10:37.930336 17538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:37.929972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.195:55035)
0809 15:10:37.930131 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:37.930209 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260809 15:10:37.930577 17536 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 15:10:38.021826 17542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:38.021031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.193:45861)
0809 15:10:38.021467 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:38.021605 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":195,"threads_started":1}
I20260809 15:10:38.485522 17547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:38.484752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.194:35593)
0809 15:10:38.485189 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:38.485300 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":213,"threads_started":1}
I20260809 15:10:38.932018 17548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:38.931104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.195:42083)
0809 15:10:38.931603 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:38.931784 (+ 181us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":256,"threads_started":1}
I20260809 15:10:39.023406 17549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:39.022657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.193:58839)
0809 15:10:39.023138 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:39.023226 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":242,"threads_started":1}
I20260809 15:10:39.487419 17550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:39.486464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.194:58099)
0809 15:10:39.487070 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:39.487186 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":329,"threads_started":1}
I20260809 15:10:39.933561 17551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:39.932840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.195:50913)
0809 15:10:39.933306 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:39.933388 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":223,"threads_started":1}
I20260809 15:10:40.024936 17552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:40.024227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.193:60327)
0809 15:10:40.024669 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:40.024754 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":223,"threads_started":1}
I20260809 15:10:40.488986 17553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:40.488282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.194:46399)
0809 15:10:40.488723 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:40.488804 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":229,"threads_started":1}
I20260809 15:10:40.935055 17554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:40.934374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.195:33427)
0809 15:10:40.934799 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:40.934879 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":225,"threads_started":1}
I20260809 15:10:41.026527 17555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:41.025785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.193:35723)
0809 15:10:41.026226 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:41.026327 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":216,"threads_started":1}
I20260809 15:10:41.490520 17558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:41.489795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.194:33225)
0809 15:10:41.490235 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:41.490322 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":221,"threads_started":1}
I20260809 15:10:41.936652 17560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:41.935901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.195:33159)
0809 15:10:41.936359 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:41.936444 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":207,"threads_started":1}
I20260809 15:10:42.028169 17563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:42.027381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.193:41349)
0809 15:10:42.027914 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:42.027995 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":291,"threads_started":1}
I20260809 15:10:42.492224 17564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:42.491387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.194:53765)
0809 15:10:42.491935 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:42.492030 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":297,"threads_started":1}
I20260809 15:10:42.938328 17567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:42.937537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.195:33389)
0809 15:10:42.938045 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:42.938135 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":247,"threads_started":1}
I20260809 15:10:43.029696 17568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:43.028979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.193:49523)
0809 15:10:43.029415 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:43.029526 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":214,"threads_started":1}
I20260809 15:10:43.493841 17569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:43.493118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.194:40441)
0809 15:10:43.493561 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:43.493658 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":229,"threads_started":1}
I20260809 15:10:43.940022 17570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:43.939197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.195:58531)
0809 15:10:43.939728 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:43.939855 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":256,"threads_started":1}
I20260809 15:10:44.031491 17574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:44.030663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.193:33049)
0809 15:10:44.031161 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:44.031249 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":211,"threads_started":1}
I20260809 15:10:44.495422 17575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:44.494657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.194:38675)
0809 15:10:44.495120 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:44.495200 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":198,"threads_started":1}
I20260809 15:10:44.941757 17577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:44.940909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.195:54727)
0809 15:10:44.941435 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:44.941520 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":226,"threads_started":1}
I20260809 15:10:45.033259 17578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:45.032495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.193:38459)
0809 15:10:45.032995 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:45.033085 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":249,"threads_started":1}
I20260809 15:10:45.496994 17579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:45.496309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.194:54195)
0809 15:10:45.496745 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:45.496830 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":223,"threads_started":1}
I20260809 15:10:45.943385 17582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:45.942618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.195:49293)
0809 15:10:45.943105 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:45.943196 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":245,"threads_started":1}
I20260809 15:10:46.034942 17584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:46.034131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.193:41437)
0809 15:10:46.034655 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:46.034742 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":268,"threads_started":1}
I20260809 15:10:46.498792 17585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:46.497883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.194:47059)
0809 15:10:46.498462 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:46.498554 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":304,"threads_started":1}
I20260809 15:10:46.945178 17586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:46.944273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.195:33363)
0809 15:10:46.944867 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:46.944958 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":277,"threads_started":1}
I20260809 15:10:47.036650 17589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:47.035913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.193:57393)
0809 15:10:47.036388 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:47.036477 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":245,"threads_started":1}
I20260809 15:10:47.500571 17590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:47.499729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.194:34763)
0809 15:10:47.500250 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:47.500346 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":269,"threads_started":1}
I20260809 15:10:47.946839 17591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:47.946095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.195:40999)
0809 15:10:47.946574 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:47.946659 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":266,"threads_started":1}
I20260809 15:10:48.038290 17592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:48.037531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.193:38047)
0809 15:10:48.038003 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:48.038098 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":245,"threads_started":1}
I20260809 15:10:48.502408 17593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:48.501652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.194:35961)
0809 15:10:48.502119 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:48.502220 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":222,"threads_started":1}
I20260809 15:10:48.948630 17595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:48.947841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.195:42945)
0809 15:10:48.948334 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:48.948441 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":246,"threads_started":1}
I20260809 15:10:49.040019 17598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:49.039170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.193:44245)
0809 15:10:49.039734 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:49.039838 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":245,"threads_started":1}
I20260809 15:10:49.504025 17599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:49.503298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.194:36731)
0809 15:10:49.503761 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:49.503862 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":228,"threads_started":1}
I20260809 15:10:49.950354 17600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:49.949594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.195:60313)
0809 15:10:49.950076 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:49.950174 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":231,"threads_started":1}
I20260809 15:10:50.041656 17603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:50.040894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.193:47171)
0809 15:10:50.041369 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:50.041464 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":222,"threads_started":1}
I20260809 15:10:50.505652 17605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:50.504874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.194:53069)
0809 15:10:50.505361 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:50.505455 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":230,"threads_started":1}
I20260809 15:10:50.952162 17607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:50.951252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.195:55965)
0809 15:10:50.951857 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:50.951959 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":267,"threads_started":1}
I20260809 15:10:51.043251 17609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:51.042477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.193:58087)
0809 15:10:51.042970 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:51.043059 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":257,"threads_started":1}
I20260809 15:10:51.507328 17610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:51.506578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.194:36083)
0809 15:10:51.507048 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:51.507131 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":262,"threads_started":1}
I20260809 15:10:51.953819 17611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:51.953077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.195:40301)
0809 15:10:51.953539 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:51.953627 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":242,"threads_started":1}
I20260809 15:10:52.045166 17612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:52.044397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.193:38229)
0809 15:10:52.044859 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:52.044966 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":231,"threads_started":1}
I20260809 15:10:52.509285 17613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:52.508419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.194:57259)
0809 15:10:52.508925 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:52.509064 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":265,"threads_started":1}
I20260809 15:10:52.938438 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 16834
I20260809 15:10:52.960381 17619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:52.959142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.195:47235)
0809 15:10:52.959772 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:52.960077 (+ 305us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":270,"threads_started":1}
I20260809 15:10:52.966536 17616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:52.939697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52182 (local address 127.24.183.193:37317)
0809 15:10:52.940337 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:52.940357 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:52.940407 (+ 50us) server_negotiation.cc:407] Connection header received
0809 15:10:52.942359 (+ 1952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:52.942372 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:52.942839 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:52.943142 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:52.944764 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:52.946048 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:52.947481 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:52.948228 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:52.948386 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:52.956278 (+ 7892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:52.956294 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:52.956321 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:52.956384 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:52.960818 (+ 4434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:52.961824 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:52.961834 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:52.961846 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:52.962099 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:52.962600 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:52.962605 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:52.962608 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:52.963431 (+ 823us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:52.965281 (+ 1850us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:52.965601 (+ 320us) server_negotiation.cc:299] Negotiation successful
0809 15:10:52.966097 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":291,"threads_started":1}
I20260809 15:10:52.968358 17096 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:10:53.066496 17622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:53.065238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:41811 (local address 127.24.183.193:50451)
0809 15:10:53.065844 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:53.066051 (+ 207us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":272,"threads_started":1}
I20260809 15:10:53.068019 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 16834
I20260809 15:10:53.096220 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 17111
I20260809 15:10:53.115005 17624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:53.097284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46724 (local address 127.24.183.194:46385)
0809 15:10:53.097811 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:53.097827 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:53.097876 (+ 49us) server_negotiation.cc:407] Connection header received
0809 15:10:53.099436 (+ 1560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:53.099446 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:53.100003 (+ 557us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:53.100343 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:53.101955 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:53.103210 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:53.104635 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:53.105313 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:53.105488 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:53.108486 (+ 2998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:53.108494 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:53.108511 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:53.108555 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:53.111591 (+ 3036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:53.112371 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:53.112379 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:53.112385 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:53.112568 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:53.112948 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:53.112951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:53.112953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:53.113498 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:53.113805 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:53.114258 (+ 453us) server_negotiation.cc:299] Negotiation successful
0809 15:10:53.114700 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":231,"threads_started":1}
I20260809 15:10:53.116573 17309 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:10:53.201916 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 17111
I20260809 15:10:53.225625 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 17320
I20260809 15:10:53.243531 17627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:53.226626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40028 (local address 127.24.183.195:36181)
0809 15:10:53.227198 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:53.227213 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:53.227257 (+ 44us) server_negotiation.cc:407] Connection header received
0809 15:10:53.228972 (+ 1715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:53.228982 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:53.229415 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:53.229754 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:53.231344 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:53.232606 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:53.233874 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:53.234524 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:53.234692 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:53.237437 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:53.237445 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:53.237460 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:53.237506 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:53.240323 (+ 2817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:53.240988 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:53.240995 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:53.241001 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:53.241171 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:53.241538 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:53.241541 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:53.241543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:53.242065 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:53.242383 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:53.242811 (+ 428us) server_negotiation.cc:299] Negotiation successful
0809 15:10:53.243214 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":257,"threads_started":1}
I20260809 15:10:53.244982 17530 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:10:53.329936 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 17320
2026-08-09T15:10:53Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24691 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:10:53 dist-test-slave-nj21 krb5kdc[17634](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:10:53 dist-test-slave-nj21 krb5kdc[17634](info): set up 2 sockets
Aug 09 15:10:53 dist-test-slave-nj21 krb5kdc[17634](info): commencing operation
krb5kdc: starting...
W20260809 15:10:54.550137 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.002s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:10:54 dist-test-slave-nj21 krb5kdc[17634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:10:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:10:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
I20260809 15:10:54.820853 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:35881
--webserver_interface=127.24.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:35177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:35881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:10:55.161049 17653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:10:55.161360 17653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:10:55.161497 17653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:10:55.168808 17653 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 15:10:55.168921 17653 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 15:10:55.168988 17653 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 15:10:55.169065 17653 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:10:55.169143 17653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:10:55.169243 17653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:10:55.169305 17653 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:10:55.169384 17653 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:10:55.179294 17653 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:35177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:35881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:35881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:10:55.180631 17653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:10:55.182034 17653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:10:55.193634 17661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:55.193595 17669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:55.193686 17662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:10:55.194588 17653 server_base.cc:1034] running on GCE node
I20260809 15:10:55.195350 17653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:10:55.196705 17653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:10:55.197930 17653 hybrid_clock.cc:648] HybridClock initialized: now 1786288255197871 us; error 25 us; skew 500 ppm
I20260809 15:10:55.198307 17653 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:10:55 dist-test-slave-nj21 krb5kdc[17634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288255, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:10:55.201502 17653 init.cc:377] Logged in from keytab as kudu/127.24.183.254@KRBTEST.COM (short username kudu)
I20260809 15:10:55.203013 17653 webserver.cc:460] Webserver started at http://127.24.183.254:36299/ using document root <none> and password file <none>
I20260809 15:10:55.203706 17653 fs_manager.cc:359] Metadata directory not provided
I20260809 15:10:55.203819 17653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:10:55.204157 17653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:10:55.207178 17653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "e1f1911f3a9b41b88d359babf89cda50"
format_stamp: "Formatted at 2026-08-09 15:10:55 on dist-test-slave-nj21"
server_key: "84af479603da0ec7563a902ff48c2210"
server_key_iv: "3412991e4a96584bea80206364fe9bd3"
server_key_version: "encryptionkey@0"
I20260809 15:10:55.208007 17653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "e1f1911f3a9b41b88d359babf89cda50"
format_stamp: "Formatted at 2026-08-09 15:10:55 on dist-test-slave-nj21"
server_key: "84af479603da0ec7563a902ff48c2210"
server_key_iv: "3412991e4a96584bea80206364fe9bd3"
server_key_version: "encryptionkey@0"
I20260809 15:10:55.213124 17653 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 15:10:55.216698 17686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:55.218019 17653 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:10:55.218173 17653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "e1f1911f3a9b41b88d359babf89cda50"
format_stamp: "Formatted at 2026-08-09 15:10:55 on dist-test-slave-nj21"
server_key: "84af479603da0ec7563a902ff48c2210"
server_key_iv: "3412991e4a96584bea80206364fe9bd3"
server_key_version: "encryptionkey@0"
I20260809 15:10:55.218374 17653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:10:55.230000 17653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:10:55.233944 17653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:10:55.234185 17653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:10:55.252636 17653 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:35881
I20260809 15:10:55.252657 17781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:35881 every 8 connection(s)
I20260809 15:10:55.253994 17653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:10:55.255342 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 17653
I20260809 15:10:55.255563 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:10:55.256470 25311 external_mini_cluster.cc:1428] Setting key ae856dbc29f024ed7c10ba05dea6083a
I20260809 15:10:55.258713 17783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 15:10:55 dist-test-slave-nj21 krb5kdc[17634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:10:55.271945 17783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50: Bootstrap starting.
I20260809 15:10:55.275436 17783 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50: Neither blocks nor log segments found. Creating new log.
I20260809 15:10:55.276592 17783 log.cc:824] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50: Log is configured to *not* fsync() on all Append() calls
I20260809 15:10:55.279886 17783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50: No bootstrap required, opened a new log
I20260809 15:10:55.280357 17788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:55.258534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56208 (local address 127.24.183.254:35881)
0809 15:10:55.259393 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:55.259408 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:55.259456 (+ 48us) server_negotiation.cc:407] Connection header received
0809 15:10:55.261632 (+ 2176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:55.261659 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:55.262147 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:55.262579 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:55.264016 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:55.266074 (+ 2058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:55.267444 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:55.268473 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:55.268651 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:55.273022 (+ 4371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:55.273030 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:55.273043 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:55.273084 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:55.276148 (+ 3064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:55.276923 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:55.276930 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:55.276936 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:55.277121 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:55.277521 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:55.277525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:55.277529 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:55.278135 (+ 606us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:55.278365 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:55.278831 (+ 466us) server_negotiation.cc:299] Negotiation successful
0809 15:10:55.279126 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":413,"threads_started":1}
I20260809 15:10:55.287925 17783 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f1911f3a9b41b88d359babf89cda50" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 35881 } }
I20260809 15:10:55.288283 17783 raft_consensus.cc:374] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:10:55.288379 17783 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f1911f3a9b41b88d359babf89cda50, State: Initialized, Role: FOLLOWER
I20260809 15:10:55.288903 17783 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f1911f3a9b41b88d359babf89cda50" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 35881 } }
I20260809 15:10:55.289117 17783 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:10:55.289291 17783 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:10:55.289505 17783 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:10:55.291875 17783 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f1911f3a9b41b88d359babf89cda50" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 35881 } }
I20260809 15:10:55.292368 17783 leader_election.cc:302] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1f1911f3a9b41b88d359babf89cda50; no voters:
I20260809 15:10:55.292924 17783 leader_election.cc:288] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:10:55.293040 17790 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:10:55.293694 17790 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [term 1 LEADER]: Becoming Leader. State: Replica: e1f1911f3a9b41b88d359babf89cda50, State: Running, Role: LEADER
I20260809 15:10:55.294255 17790 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f1911f3a9b41b88d359babf89cda50" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 35881 } }
I20260809 15:10:55.294905 17783 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:10:55.297062 17796 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1f1911f3a9b41b88d359babf89cda50. Latest consensus state: current_term: 1 leader_uuid: "e1f1911f3a9b41b88d359babf89cda50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f1911f3a9b41b88d359babf89cda50" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 35881 } } }
I20260809 15:10:55.297540 17796 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [sys.catalog]: This master's current role is: LEADER
I20260809 15:10:55.297086 17794 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e1f1911f3a9b41b88d359babf89cda50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f1911f3a9b41b88d359babf89cda50" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 35881 } } }
I20260809 15:10:55.298144 17794 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [sys.catalog]: This master's current role is: LEADER
I20260809 15:10:55.300484 17802 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:10:55.306108 17802 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:10:55.314616 17802 catalog_manager.cc:1347] Generated new cluster ID: 9ff381450f124676b3c29c4fed0a7ac4
I20260809 15:10:55.314718 17802 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:10:55.324245 17802 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:10:55.325507 17802 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:10:55.336242 17802 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50: Generated new TSK 0
I20260809 15:10:55.337050 17802 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab.
Entry for principal kudu/127.24.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab.
I20260809 15:10:55.409025 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.193:0
--local_ip_for_outbound_sockets=127.24.183.193
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:35881
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:35177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:10:55.773073 17829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:10:55.773342 17829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:10:55.773465 17829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:10:55.780858 17829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:10:55.781000 17829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:10:55.781139 17829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.193
I20260809 15:10:55.790952 17829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:35177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:35881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.193
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:10:55.792260 17829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:10:55.793574 17829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:10:55.806727 17844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:55.806667 17841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:55.806717 17840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:10:55.809551 17829 server_base.cc:1034] running on GCE node
I20260809 15:10:55.810295 17829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:10:55.815411 17829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:10:55.816592 17829 hybrid_clock.cc:648] HybridClock initialized: now 1786288255816526 us; error 60 us; skew 500 ppm
I20260809 15:10:55.816905 17829 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:10:55 dist-test-slave-nj21 krb5kdc[17634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288255, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:10:55.819970 17829 init.cc:377] Logged in from keytab as kudu/127.24.183.193@KRBTEST.COM (short username kudu)
I20260809 15:10:55.821485 17829 webserver.cc:460] Webserver started at http://127.24.183.193:41175/ using document root <none> and password file <none>
I20260809 15:10:55.822227 17829 fs_manager.cc:359] Metadata directory not provided
I20260809 15:10:55.822347 17829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:10:55.822641 17829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:10:55.825642 17829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/data/instance:
uuid: "e6fbb915997b4f3da1537556271642f6"
format_stamp: "Formatted at 2026-08-09 15:10:55 on dist-test-slave-nj21"
server_key: "12c0b4701e3adf4c3c503c274d80b43a"
server_key_iv: "44e535ee8626ae52197aabd6f2d29f1d"
server_key_version: "encryptionkey@0"
I20260809 15:10:55.826387 17829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance:
uuid: "e6fbb915997b4f3da1537556271642f6"
format_stamp: "Formatted at 2026-08-09 15:10:55 on dist-test-slave-nj21"
server_key: "12c0b4701e3adf4c3c503c274d80b43a"
server_key_iv: "44e535ee8626ae52197aabd6f2d29f1d"
server_key_version: "encryptionkey@0"
I20260809 15:10:55.831404 17829 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 15:10:55.834736 17855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:55.836196 17829 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:10:55.836376 17829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/wal
uuid: "e6fbb915997b4f3da1537556271642f6"
format_stamp: "Formatted at 2026-08-09 15:10:55 on dist-test-slave-nj21"
server_key: "12c0b4701e3adf4c3c503c274d80b43a"
server_key_iv: "44e535ee8626ae52197aabd6f2d29f1d"
server_key_version: "encryptionkey@0"
I20260809 15:10:55.836570 17829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:10:55.863081 17829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:10:55.867153 17829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:10:55.867441 17829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:10:55.868432 17829 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:10:55.869834 17829 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:10:55.869916 17829 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:55.870038 17829 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:10:55.870100 17829 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:55.902406 17829 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.193:46463
I20260809 15:10:55.902459 18038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.193:46463 every 8 connection(s)
I20260809 15:10:55.903703 17829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
I20260809 15:10:55.908327 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 17829
I20260809 15:10:55.908528 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance
I20260809 15:10:55.909168 25311 external_mini_cluster.cc:1428] Setting key 38ea9e5a3410f566167a160d67aa9e10
Aug 09 15:10:55 dist-test-slave-nj21 krb5kdc[17634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288255, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:10:55.923935 18042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:55.906190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:57067 (local address 127.24.183.254:35881)
0809 15:10:55.906665 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:55.906670 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:55.907472 (+ 802us) server_negotiation.cc:407] Connection header received
0809 15:10:55.909464 (+ 1992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:55.909470 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:55.909552 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:55.909653 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:55.911364 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:55.912148 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:55.913669 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:55.914089 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:55.914363 (+ 274us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:55.918414 (+ 4051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:55.918419 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:55.918422 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:55.918456 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:55.920872 (+ 2416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:55.921603 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:55.921609 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:55.921613 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:55.921750 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:55.922172 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:55.922178 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:55.922181 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:55.922518 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:55.922655 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:55.923412 (+ 757us) server_negotiation.cc:299] Negotiation successful
0809 15:10:55.923593 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":266,"threads_started":1}
I20260809 15:10:55.925206 18045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:55.906616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.193:57067)
0809 15:10:55.907318 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:55.907359 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:55.909168 (+ 1809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:55.909894 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:55.909902 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:55.910545 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:10:55.911194 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:55.911211 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:55.912301 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:55.912305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:55.913526 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:55.913544 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:55.914461 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:55.914470 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:55.914717 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:55.915467 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:10:55.915497 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:10:55.918198 (+ 2701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:10:55.921003 (+ 2805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:55.921013 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:55.921028 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:55.921487 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:55.921856 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:55.921861 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:55.921865 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:55.922057 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:55.922638 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:10:55.922646 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:10:55.923004 (+ 358us) client_negotiation.cc:769] Sending connection context
0809 15:10:55.923274 (+ 270us) client_negotiation.cc:240] Negotiation successful
0809 15:10:55.923842 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":266,"threads_started":1}
I20260809 15:10:55.926740 18039 heartbeater.cc:347] Connected to a master server at 127.24.183.254:35881
I20260809 15:10:55.927031 18039 heartbeater.cc:464] Registering TS with master...
I20260809 15:10:55.927785 18039 heartbeater.cc:511] Master 127.24.183.254:35881 requested a full tablet report, sending...
I20260809 15:10:55.929836 17717 ts_manager.cc:194] Registered new tserver with Master: e6fbb915997b4f3da1537556271642f6 (127.24.183.193:46463)
I20260809 15:10:55.931293 17717 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.193@KRBTEST.COM'} at 127.24.183.193:57067
WARNING: no policy specified for kudu/127.24.183.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab.
Entry for principal kudu/127.24.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab.
I20260809 15:10:55.973001 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.194:0
--local_ip_for_outbound_sockets=127.24.183.194
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:35881
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:35177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:10:56.313838 18050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:10:56.314096 18050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:10:56.314225 18050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:10:56.321272 18050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:10:56.321405 18050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:10:56.321570 18050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.194
I20260809 15:10:56.331230 18050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:35177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:35881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.194
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:10:56.332546 18050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:10:56.333889 18050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:10:56.347504 18058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:56.347481 18062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:56.347463 18060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:10:56.348538 18050 server_base.cc:1034] running on GCE node
I20260809 15:10:56.349205 18050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:10:56.350214 18050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:10:56.351508 18050 hybrid_clock.cc:648] HybridClock initialized: now 1786288256351422 us; error 75 us; skew 500 ppm
I20260809 15:10:56.351861 18050 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:10:56 dist-test-slave-nj21 krb5kdc[17634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288256, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:10:56.355026 18050 init.cc:377] Logged in from keytab as kudu/127.24.183.194@KRBTEST.COM (short username kudu)
I20260809 15:10:56.356500 18050 webserver.cc:460] Webserver started at http://127.24.183.194:36033/ using document root <none> and password file <none>
I20260809 15:10:56.357170 18050 fs_manager.cc:359] Metadata directory not provided
I20260809 15:10:56.357296 18050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:10:56.357599 18050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:10:56.360551 18050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/data/instance:
uuid: "2ee1436310174f20aa6e60f0e644cc93"
format_stamp: "Formatted at 2026-08-09 15:10:56 on dist-test-slave-nj21"
server_key: "5b45486b0132c29cdb81515d61c07361"
server_key_iv: "86e6e5440eda4ded60d4be036963a159"
server_key_version: "encryptionkey@0"
I20260809 15:10:56.361297 18050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance:
uuid: "2ee1436310174f20aa6e60f0e644cc93"
format_stamp: "Formatted at 2026-08-09 15:10:56 on dist-test-slave-nj21"
server_key: "5b45486b0132c29cdb81515d61c07361"
server_key_iv: "86e6e5440eda4ded60d4be036963a159"
server_key_version: "encryptionkey@0"
I20260809 15:10:56.366482 18050 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 15:10:56.370132 18074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:56.371438 18050 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 15:10:56.371589 18050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/wal
uuid: "2ee1436310174f20aa6e60f0e644cc93"
format_stamp: "Formatted at 2026-08-09 15:10:56 on dist-test-slave-nj21"
server_key: "5b45486b0132c29cdb81515d61c07361"
server_key_iv: "86e6e5440eda4ded60d4be036963a159"
server_key_version: "encryptionkey@0"
I20260809 15:10:56.371840 18050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:10:56.393550 18050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:10:56.397703 18050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:10:56.397966 18050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:10:56.398978 18050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:10:56.400321 18050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:10:56.400393 18050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:56.400514 18050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:10:56.400565 18050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:56.432817 18050 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.194:42299
I20260809 15:10:56.432920 18263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.194:42299 every 8 connection(s)
I20260809 15:10:56.434077 18050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
I20260809 15:10:56.440255 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 18050
I20260809 15:10:56.440483 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance
I20260809 15:10:56.441195 25311 external_mini_cluster.cc:1428] Setting key 716f62412b18e8b6f1ab7b774bea594b
Aug 09 15:10:56 dist-test-slave-nj21 krb5kdc[17634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288256, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:10:56.455000 18270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:56.436576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:57611 (local address 127.24.183.254:35881)
0809 15:10:56.437026 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:56.437031 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:56.437862 (+ 831us) server_negotiation.cc:407] Connection header received
0809 15:10:56.439732 (+ 1870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:56.439738 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:56.439819 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:56.439941 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:56.441974 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:56.442757 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:56.444820 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:56.445249 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:56.445355 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:56.449257 (+ 3902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:56.449266 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:56.449270 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:56.449304 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:56.451950 (+ 2646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:56.452752 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:56.452757 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:56.452759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:56.452890 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:56.453339 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:56.453342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:56.453344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:56.453610 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:56.453751 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:56.454546 (+ 795us) server_negotiation.cc:299] Negotiation successful
0809 15:10:56.454730 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":231,"threads_started":1}
I20260809 15:10:56.456914 18271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:56.436979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.194:57611)
0809 15:10:56.437698 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:56.437742 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:56.439438 (+ 1696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:56.440222 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:56.440255 (+ 33us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:56.440983 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:10:56.441786 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:56.441815 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:56.442914 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:56.442920 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:56.444682 (+ 1762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:56.444698 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:56.445482 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:56.445488 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:56.445775 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:56.446450 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:10:56.446477 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:10:56.449003 (+ 2526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:10:56.452092 (+ 3089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:56.452100 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:56.452113 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:56.452635 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:56.452991 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:56.452995 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:56.452999 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:56.453227 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:56.453768 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:10:56.453776 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:10:56.454164 (+ 388us) client_negotiation.cc:769] Sending connection context
0809 15:10:56.454465 (+ 301us) client_negotiation.cc:240] Negotiation successful
0809 15:10:56.454955 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":284,"threads_started":1}
I20260809 15:10:56.458396 18265 heartbeater.cc:347] Connected to a master server at 127.24.183.254:35881
I20260809 15:10:56.458691 18265 heartbeater.cc:464] Registering TS with master...
I20260809 15:10:56.459405 18265 heartbeater.cc:511] Master 127.24.183.254:35881 requested a full tablet report, sending...
I20260809 15:10:56.460999 17717 ts_manager.cc:194] Registered new tserver with Master: 2ee1436310174f20aa6e60f0e644cc93 (127.24.183.194:42299)
I20260809 15:10:56.462081 17717 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.194@KRBTEST.COM'} at 127.24.183.194:57611
WARNING: no policy specified for kudu/127.24.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab.
Entry for principal kudu/127.24.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab.
I20260809 15:10:56.502867 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.195:0
--local_ip_for_outbound_sockets=127.24.183.195
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:35881
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:35177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:10:56.840296 18279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:10:56.840549 18279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:10:56.840646 18279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:10:56.847642 18279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:10:56.847786 18279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:10:56.847913 18279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.195
I20260809 15:10:56.857280 18279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:35177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:35881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.195
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:10:56.858534 18279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:10:56.859836 18279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:10:56.872280 18286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:56.872396 18292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:10:56.872466 18288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:10:56.873688 18279 server_base.cc:1034] running on GCE node
I20260809 15:10:56.874248 18279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:10:56.875209 18279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:10:56.876406 18279 hybrid_clock.cc:648] HybridClock initialized: now 1786288256876360 us; error 33 us; skew 500 ppm
I20260809 15:10:56.876711 18279 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:10:56 dist-test-slave-nj21 krb5kdc[17634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288256, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:10:56.879999 18279 init.cc:377] Logged in from keytab as kudu/127.24.183.195@KRBTEST.COM (short username kudu)
I20260809 15:10:56.881441 18279 webserver.cc:460] Webserver started at http://127.24.183.195:44075/ using document root <none> and password file <none>
I20260809 15:10:56.882109 18279 fs_manager.cc:359] Metadata directory not provided
I20260809 15:10:56.882225 18279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:10:56.882514 18279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:10:56.885640 18279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/data/instance:
uuid: "d40021626873405fb8183692867ac93f"
format_stamp: "Formatted at 2026-08-09 15:10:56 on dist-test-slave-nj21"
server_key: "ea035a8094f632db2b52fe36227f7a2e"
server_key_iv: "fc5815666a05ebee49041246e29646ae"
server_key_version: "encryptionkey@0"
I20260809 15:10:56.886433 18279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance:
uuid: "d40021626873405fb8183692867ac93f"
format_stamp: "Formatted at 2026-08-09 15:10:56 on dist-test-slave-nj21"
server_key: "ea035a8094f632db2b52fe36227f7a2e"
server_key_iv: "fc5815666a05ebee49041246e29646ae"
server_key_version: "encryptionkey@0"
I20260809 15:10:56.891551 18279 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 15:10:56.894912 18299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:56.896384 18279 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:10:56.896531 18279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/wal
uuid: "d40021626873405fb8183692867ac93f"
format_stamp: "Formatted at 2026-08-09 15:10:56 on dist-test-slave-nj21"
server_key: "ea035a8094f632db2b52fe36227f7a2e"
server_key_iv: "fc5815666a05ebee49041246e29646ae"
server_key_version: "encryptionkey@0"
I20260809 15:10:56.896699 18279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:10:56.917387 18279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:10:56.922070 18279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:10:56.922344 18279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:10:56.923322 18279 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:10:56.924643 18279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:10:56.924721 18279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:56.924842 18279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:10:56.924903 18279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:10:56.934161 18039 heartbeater.cc:503] Master 127.24.183.254:35881 was elected leader, sending a full tablet report...
I20260809 15:10:56.960304 18279 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.195:38479
I20260809 15:10:56.960346 18493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.195:38479 every 8 connection(s)
I20260809 15:10:56.961557 18279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
I20260809 15:10:56.970211 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 18279
I20260809 15:10:56.970410 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance
I20260809 15:10:56.971024 25311 external_mini_cluster.cc:1428] Setting key c02970aabedc18f10178d41c08555004
Aug 09 15:10:56 dist-test-slave-nj21 krb5kdc[17634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288256, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:10:56.981249 18496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:56.963967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:34261 (local address 127.24.183.254:35881)
0809 15:10:56.964412 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:56.964416 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:56.965305 (+ 889us) server_negotiation.cc:407] Connection header received
0809 15:10:56.967142 (+ 1837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:56.967148 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:56.967229 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:56.967329 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:56.968848 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:56.969666 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:56.971234 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:56.971747 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:56.971868 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:56.975660 (+ 3792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:56.975664 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:56.975696 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:56.975741 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:56.978222 (+ 2481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:56.978971 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:56.978975 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:56.978977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:56.979100 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:56.979513 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:56.979518 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:56.979520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:56.979835 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:56.979986 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:56.980645 (+ 659us) server_negotiation.cc:299] Negotiation successful
0809 15:10:56.980881 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":231,"threads_started":1}
I20260809 15:10:56.982110 18497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:56.964423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.195:34261)
0809 15:10:56.965132 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:56.965179 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:56.966877 (+ 1698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:56.967570 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:56.967577 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:56.968120 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:10:56.968677 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:56.968694 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:56.969818 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:56.969822 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:56.971093 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:56.971108 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:56.972039 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:56.972048 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:56.972337 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:56.973036 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:10:56.973067 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:10:56.975468 (+ 2401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:10:56.978352 (+ 2884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:56.978361 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:56.978375 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:56.978854 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:56.979202 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:10:56.979208 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:10:56.979212 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:10:56.979404 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:10:56.979960 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:10:56.979968 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:10:56.980296 (+ 328us) client_negotiation.cc:769] Sending connection context
0809 15:10:56.980532 (+ 236us) client_negotiation.cc:240] Negotiation successful
0809 15:10:56.981026 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":275,"threads_started":1}
I20260809 15:10:56.983362 18494 heartbeater.cc:347] Connected to a master server at 127.24.183.254:35881
I20260809 15:10:56.983652 18494 heartbeater.cc:464] Registering TS with master...
I20260809 15:10:56.984452 18494 heartbeater.cc:511] Master 127.24.183.254:35881 requested a full tablet report, sending...
I20260809 15:10:56.985935 17717 ts_manager.cc:194] Registered new tserver with Master: d40021626873405fb8183692867ac93f (127.24.183.195:38479)
I20260809 15:10:56.986661 25311 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:10:56.986860 17717 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.195@KRBTEST.COM'} at 127.24.183.195:34261
I20260809 15:10:57.004127 18496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:56.991879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56212 (local address 127.24.183.254:35881)
0809 15:10:56.992073 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:56.992080 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:56.992505 (+ 425us) server_negotiation.cc:407] Connection header received
0809 15:10:56.992618 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:56.992622 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:56.992679 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:56.992754 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:56.994128 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:56.994839 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:56.996532 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:56.996919 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:56.997007 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:56.999213 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:56.999218 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:56.999221 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:56.999249 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:57.001730 (+ 2481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:57.002363 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:57.002367 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:57.002369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:57.002493 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:57.002788 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:57.002792 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:57.002794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:57.003203 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:57.003402 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:57.003620 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 15:10:57.003827 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 15:10:57.007951 17717 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56212:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:10:57.014966 17717 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:10:57.037001 18513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:57.026883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:46463 (local address 127.0.0.1:44692)
0809 15:10:57.027603 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:57.027655 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:57.027854 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:57.029041 (+ 1187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:57.029052 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:57.029077 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:10:57.029519 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:57.029540 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:57.031514 (+ 1974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:57.031520 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:57.034093 (+ 2573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:57.034107 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:57.035963 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:57.035970 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:57.036173 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:57.036229 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 15:10:57.036333 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 15:10:57.036529 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"mutex_wait_us":82,"thread_start_us":297,"threads_started":1}
I20260809 15:10:57.037595 18517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:57.028088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53896 (local address 127.24.183.194:42299)
0809 15:10:57.028572 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:57.028580 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:57.029167 (+ 587us) server_negotiation.cc:407] Connection header received
0809 15:10:57.029294 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:57.029300 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:57.029528 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:57.029708 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:10:57.030585 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:57.032048 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:57.034627 (+ 2579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:57.036019 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:57.036170 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:57.036371 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:57.037102 (+ 731us) server_negotiation.cc:299] Negotiation successful
0809 15:10:57.037289 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":337,"threads_started":1}
I20260809 15:10:57.038125 18515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:57.027871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:42299 (local address 127.0.0.1:53896)
0809 15:10:57.028894 (+ 1023us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:57.028907 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:57.029026 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:57.029945 (+ 919us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:57.029949 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:57.029975 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:10:57.030393 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:57.030406 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:57.032213 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:57.032218 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:57.034452 (+ 2234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:57.034466 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:57.036514 (+ 2048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:57.036520 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:57.036695 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:57.036730 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 15:10:57.036814 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 15:10:57.036945 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":928,"mutex_wait_us":22,"thread_start_us":795,"threads_started":1}
I20260809 15:10:57.038268 18514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:57.027056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44692 (local address 127.24.183.193:46463)
0809 15:10:57.028400 (+ 1344us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:57.028410 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:57.028443 (+ 33us) server_negotiation.cc:407] Connection header received
0809 15:10:57.028556 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:57.028564 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:57.028822 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:57.029040 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:10:57.029748 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:57.031356 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:57.034277 (+ 2921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:57.035731 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:57.037548 (+ 1817us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:57.037721 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:57.037810 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 15:10:57.037969 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1101,"thread_start_us":1138,"threads_started":1}
I20260809 15:10:57.042083 18158 tablet_service.cc:1467] Processing CreateTablet for tablet f66eedcf6ed34dbba91712384518f6f2 (DEFAULT_TABLE table=test-table [id=48624c77f2aa4a45b073dfea786d2584]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:10:57.042676 17935 tablet_service.cc:1467] Processing CreateTablet for tablet f66eedcf6ed34dbba91712384518f6f2 (DEFAULT_TABLE table=test-table [id=48624c77f2aa4a45b073dfea786d2584]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:10:57.043519 18158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f66eedcf6ed34dbba91712384518f6f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:10:57.043996 17935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f66eedcf6ed34dbba91712384518f6f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:10:57.046995 18513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:57.029627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:38479 (local address 127.0.0.1:54788)
0809 15:10:57.037226 (+ 7599us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:57.037243 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:57.037331 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:57.040706 (+ 3375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:57.040711 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:57.040736 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:10:57.040888 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:57.040899 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:57.042849 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:57.042854 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:57.044548 (+ 1694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:57.044563 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:57.046256 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:57.046262 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:57.046441 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:57.046474 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 15:10:57.046567 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 15:10:57.046705 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7512,"thread_start_us":9319,"threads_started":1}
I20260809 15:10:57.047348 18521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:57.029926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54788 (local address 127.24.183.195:38479)
0809 15:10:57.040055 (+ 10129us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:57.040077 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:57.040117 (+ 40us) server_negotiation.cc:407] Connection header received
0809 15:10:57.040244 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:57.040253 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:57.040500 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:57.040703 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:10:57.041038 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:57.042694 (+ 1656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:57.044720 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:57.046086 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:57.046272 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:57.046442 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:57.046642 (+ 200us) server_negotiation.cc:299] Negotiation successful
0809 15:10:57.046842 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9844,"thread_start_us":9678,"threads_started":1}
I20260809 15:10:57.051474 18385 tablet_service.cc:1467] Processing CreateTablet for tablet f66eedcf6ed34dbba91712384518f6f2 (DEFAULT_TABLE table=test-table [id=48624c77f2aa4a45b073dfea786d2584]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:10:57.052961 18385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f66eedcf6ed34dbba91712384518f6f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:10:57.055629 18523 tablet_bootstrap.cc:492] T f66eedcf6ed34dbba91712384518f6f2 P 2ee1436310174f20aa6e60f0e644cc93: Bootstrap starting.
I20260809 15:10:57.057502 18524 tablet_bootstrap.cc:492] T f66eedcf6ed34dbba91712384518f6f2 P e6fbb915997b4f3da1537556271642f6: Bootstrap starting.
I20260809 15:10:57.059391 18523 tablet_bootstrap.cc:654] T f66eedcf6ed34dbba91712384518f6f2 P 2ee1436310174f20aa6e60f0e644cc93: Neither blocks nor log segments found. Creating new log.
I20260809 15:10:57.060698 18523 log.cc:824] T f66eedcf6ed34dbba91712384518f6f2 P 2ee1436310174f20aa6e60f0e644cc93: Log is configured to *not* fsync() on all Append() calls
I20260809 15:10:57.061337 18524 tablet_bootstrap.cc:654] T f66eedcf6ed34dbba91712384518f6f2 P e6fbb915997b4f3da1537556271642f6: Neither blocks nor log segments found. Creating new log.
I20260809 15:10:57.062353 18524 log.cc:824] T f66eedcf6ed34dbba91712384518f6f2 P e6fbb915997b4f3da1537556271642f6: Log is configured to *not* fsync() on all Append() calls
I20260809 15:10:57.063880 18523 tablet_bootstrap.cc:492] T f66eedcf6ed34dbba91712384518f6f2 P 2ee1436310174f20aa6e60f0e644cc93: No bootstrap required, opened a new log
I20260809 15:10:57.064157 18523 ts_tablet_manager.cc:1397] T f66eedcf6ed34dbba91712384518f6f2 P 2ee1436310174f20aa6e60f0e644cc93: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260809 15:10:57.065320 18524 tablet_bootstrap.cc:492] T f66eedcf6ed34dbba91712384518f6f2 P e6fbb915997b4f3da1537556271642f6: No bootstrap required, opened a new log
I20260809 15:10:57.065584 18524 ts_tablet_manager.cc:1397] T f66eedcf6ed34dbba91712384518f6f2 P e6fbb915997b4f3da1537556271642f6: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 15:10:57.065904 18528 tablet_bootstrap.cc:492] T f66eedcf6ed34dbba91712384518f6f2 P d40021626873405fb8183692867ac93f: Bootstrap starting.
I20260809 15:10:57.068922 18528 tablet_bootstrap.cc:654] T f66eedcf6ed34dbba91712384518f6f2 P d40021626873405fb8183692867ac93f: Neither blocks nor log segments found. Creating new log.
I20260809 15:10:57.070160 18528 log.cc:824] T f66eedcf6ed34dbba91712384518f6f2 P d40021626873405fb8183692867ac93f: Log is configured to *not* fsync() on all Append() calls
I20260809 15:10:57.072566 18528 tablet_bootstrap.cc:492] T f66eedcf6ed34dbba91712384518f6f2 P d40021626873405fb8183692867ac93f: No bootstrap required, opened a new log
I20260809 15:10:57.072824 18528 ts_tablet_manager.cc:1397] T f66eedcf6ed34dbba91712384518f6f2 P d40021626873405fb8183692867ac93f: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260809 15:10:57.075284 18523 raft_consensus.cc:348] T f66eedcf6ed34dbba91712384518f6f2 P 2ee1436310174f20aa6e60f0e644cc93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6fbb915997b4f3da1537556271642f6" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 46463 } } peers { permanent_uuid: "2ee1436310174f20aa6e60f0e644cc93" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 42299 } } peers { permanent_uuid: "d40021626873405fb8183692867ac93f" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38479 } }
I20260809 15:10:57.075786 18523 raft_consensus.cc:374] T f66eedcf6ed34dbba91712384518f6f2 P 2ee1436310174f20aa6e60f0e644cc93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:10:57.075896 18523 raft_consensus.cc:729] T f66eedcf6ed34dbba91712384518f6f2 P 2ee1436310174f20aa6e60f0e644cc93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ee1436310174f20aa6e60f0e644cc93, State: Initialized, Role: FOLLOWER
I20260809 15:10:57.075940 18524 raft_consensus.cc:348] T f66eedcf6ed34dbba91712384518f6f2 P e6fbb915997b4f3da1537556271642f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6fbb915997b4f3da1537556271642f6" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 46463 } } peers { permanent_uuid: "2ee1436310174f20aa6e60f0e644cc93" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 42299 } } peers { permanent_uuid: "d40021626873405fb8183692867ac93f" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38479 } }
I20260809 15:10:57.076354 18524 raft_consensus.cc:374] T f66eedcf6ed34dbba91712384518f6f2 P e6fbb915997b4f3da1537556271642f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:10:57.076459 18524 raft_consensus.cc:729] T f66eedcf6ed34dbba91712384518f6f2 P e6fbb915997b4f3da1537556271642f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6fbb915997b4f3da1537556271642f6, State: Initialized, Role: FOLLOWER
I20260809 15:10:57.076519 18523 consensus_queue.cc:260] T f66eedcf6ed34dbba91712384518f6f2 P 2ee1436310174f20aa6e60f0e644cc93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6fbb915997b4f3da1537556271642f6" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 46463 } } peers { permanent_uuid: "2ee1436310174f20aa6e60f0e644cc93" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 42299 } } peers { permanent_uuid: "d40021626873405fb8183692867ac93f" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38479 } }
I20260809 15:10:57.077016 18524 consensus_queue.cc:260] T f66eedcf6ed34dbba91712384518f6f2 P e6fbb915997b4f3da1537556271642f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6fbb915997b4f3da1537556271642f6" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 46463 } } peers { permanent_uuid: "2ee1436310174f20aa6e60f0e644cc93" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 42299 } } peers { permanent_uuid: "d40021626873405fb8183692867ac93f" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38479 } }
I20260809 15:10:57.077685 18265 heartbeater.cc:503] Master 127.24.183.254:35881 was elected leader, sending a full tablet report...
I20260809 15:10:57.078375 18523 ts_tablet_manager.cc:1428] T f66eedcf6ed34dbba91712384518f6f2 P 2ee1436310174f20aa6e60f0e644cc93: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260809 15:10:57.080148 18524 ts_tablet_manager.cc:1428] T f66eedcf6ed34dbba91712384518f6f2 P e6fbb915997b4f3da1537556271642f6: Time spent starting tablet: real 0.014s user 0.012s sys 0.004s
I20260809 15:10:57.081575 18528 raft_consensus.cc:348] T f66eedcf6ed34dbba91712384518f6f2 P d40021626873405fb8183692867ac93f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6fbb915997b4f3da1537556271642f6" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 46463 } } peers { permanent_uuid: "2ee1436310174f20aa6e60f0e644cc93" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 42299 } } peers { permanent_uuid: "d40021626873405fb8183692867ac93f" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38479 } }
I20260809 15:10:57.081940 18528 raft_consensus.cc:374] T f66eedcf6ed34dbba91712384518f6f2 P d40021626873405fb8183692867ac93f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:10:57.082048 18528 raft_consensus.cc:729] T f66eedcf6ed34dbba91712384518f6f2 P d40021626873405fb8183692867ac93f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d40021626873405fb8183692867ac93f, State: Initialized, Role: FOLLOWER
I20260809 15:10:57.082742 18528 consensus_queue.cc:260] T f66eedcf6ed34dbba91712384518f6f2 P d40021626873405fb8183692867ac93f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6fbb915997b4f3da1537556271642f6" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 46463 } } peers { permanent_uuid: "2ee1436310174f20aa6e60f0e644cc93" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 42299 } } peers { permanent_uuid: "d40021626873405fb8183692867ac93f" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38479 } }
I20260809 15:10:57.083897 18494 heartbeater.cc:503] Master 127.24.183.254:35881 was elected leader, sending a full tablet report...
I20260809 15:10:57.084460 18528 ts_tablet_manager.cc:1428] T f66eedcf6ed34dbba91712384518f6f2 P d40021626873405fb8183692867ac93f: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
W20260809 15:10:57.156039 18040 tablet.cc:2369] T f66eedcf6ed34dbba91712384518f6f2 P e6fbb915997b4f3da1537556271642f6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:10:57.178115 18530 raft_consensus.cc:482] T f66eedcf6ed34dbba91712384518f6f2 P 2ee1436310174f20aa6e60f0e644cc93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:10:57.178465 18530 raft_consensus.cc:504] T f66eedcf6ed34dbba91712384518f6f2 P 2ee1436310174f20aa6e60f0e644cc93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6fbb915997b4f3da1537556271642f6" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 46463 } } peers { permanent_uuid: "2ee1436310174f20aa6e60f0e644cc93" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 42299 } } peers { permanent_uuid: "d40021626873405fb8183692867ac93f" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38479 } }
I20260809 15:10:57.180393 18530 leader_election.cc:288] T f66eedcf6ed34dbba91712384518f6f2 P 2ee1436310174f20aa6e60f0e644cc93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6fbb915997b4f3da1537556271642f6 (127.24.183.193:46463), d40021626873405fb8183692867ac93f (127.24.183.195:38479)
W20260809 15:10:57.185992 18269 tablet.cc:2369] T f66eedcf6ed34dbba91712384518f6f2 P 2ee1436310174f20aa6e60f0e644cc93: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:10:57.188054 18514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:57.180808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:40901 (local address 127.24.183.193:46463)
0809 15:10:57.181007 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:57.181012 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:57.181671 (+ 659us) server_negotiation.cc:407] Connection header received
0809 15:10:57.181902 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:57.181909 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:57.181995 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:57.182117 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:10:57.182894 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:57.183900 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:57.185906 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:57.187141 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:57.187337 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:57.187426 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:57.187724 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 15:10:57.187838 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 15:10:57.188244 18533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:57.180827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:46463 (local address 127.24.183.194:40901)
0809 15:10:57.181536 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:57.181557 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:57.181710 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:57.182240 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:57.182245 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:57.182281 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:10:57.182708 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:57.182723 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:57.184042 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:57.184048 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:57.185733 (+ 1685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:57.185747 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:57.187368 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:57.187373 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:57.187511 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:57.187547 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 15:10:57.187654 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 15:10:57.187830 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"mutex_wait_us":29,"thread_start_us":372,"threads_started":1}
I20260809 15:10:57.188495 18521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:57.181897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:33147 (local address 127.24.183.195:38479)
0809 15:10:57.182122 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:57.182127 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:57.182387 (+ 260us) server_negotiation.cc:407] Connection header received
0809 15:10:57.182594 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:57.182600 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:57.182695 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:57.182809 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:10:57.183456 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:57.184259 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:57.186173 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:57.187430 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:57.187577 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:57.187640 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:57.188069 (+ 429us) server_negotiation.cc:299] Negotiation successful
0809 15:10:57.188214 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 15:10:57.188795 18536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:57.181666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:38479 (local address 127.24.183.194:33147)
0809 15:10:57.182270 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0809 15:10:57.182287 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:10:57.182448 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:10:57.182837 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:10:57.182842 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:10:57.182868 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:10:57.183249 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:57.183262 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:57.184416 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:57.184419 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:57.186001 (+ 1582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:10:57.186016 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:10:57.187648 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:10:57.187654 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:10:57.187885 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:57.187919 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 15:10:57.188013 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 15:10:57.188428 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"mutex_wait_us":25,"thread_start_us":293,"threads_started":1}
I20260809 15:10:57.189388 17970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f66eedcf6ed34dbba91712384518f6f2" candidate_uuid: "2ee1436310174f20aa6e60f0e644cc93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6fbb915997b4f3da1537556271642f6" is_pre_election: true
I20260809 15:10:57.189788 18415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f66eedcf6ed34dbba91712384518f6f2" candidate_uuid: "2ee1436310174f20aa6e60f0e644cc93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d40021626873405fb8183692867ac93f" is_pre_election: true
I20260809 15:10:57.190099 17970 raft_consensus.cc:2454] T f66eedcf6ed34dbba91712384518f6f2 P e6fbb915997b4f3da1537556271642f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ee1436310174f20aa6e60f0e644cc93 in term 0.
I20260809 15:10:57.190434 18415 raft_consensus.cc:2454] T f66eedcf6ed34dbba91712384518f6f2 P d40021626873405fb8183692867ac93f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ee1436310174f20aa6e60f0e644cc93 in term 0.
I20260809 15:10:57.190863 18076 leader_election.cc:302] T f66eedcf6ed34dbba91712384518f6f2 P 2ee1436310174f20aa6e60f0e644cc93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ee1436310174f20aa6e60f0e644cc93, e6fbb915997b4f3da1537556271642f6; no voters:
I20260809 15:10:57.191277 18530 raft_consensus.cc:2781] T f66eedcf6ed34dbba91712384518f6f2 P 2ee1436310174f20aa6e60f0e644cc93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:10:57.191416 18530 raft_consensus.cc:482] T f66eedcf6ed34dbba91712384518f6f2 P 2ee1436310174f20aa6e60f0e644cc93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:10:57.191531 18530 raft_consensus.cc:3037] T f66eedcf6ed34dbba91712384518f6f2 P 2ee1436310174f20aa6e60f0e644cc93 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:10:57.193856 18530 raft_consensus.cc:504] T f66eedcf6ed34dbba91712384518f6f2 P 2ee1436310174f20aa6e60f0e644cc93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6fbb915997b4f3da1537556271642f6" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 46463 } } peers { permanent_uuid: "2ee1436310174f20aa6e60f0e644cc93" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 42299 } } peers { permanent_uuid: "d40021626873405fb8183692867ac93f" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38479 } }
I20260809 15:10:57.194661 18530 leader_election.cc:288] T f66eedcf6ed34dbba91712384518f6f2 P 2ee1436310174f20aa6e60f0e644cc93 [CANDIDATE]: Term 1 election: Requested vote from peers e6fbb915997b4f3da1537556271642f6 (127.24.183.193:46463), d40021626873405fb8183692867ac93f (127.24.183.195:38479)
I20260809 15:10:57.195148 17970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f66eedcf6ed34dbba91712384518f6f2" candidate_uuid: "2ee1436310174f20aa6e60f0e644cc93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6fbb915997b4f3da1537556271642f6"
I20260809 15:10:57.195250 18415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f66eedcf6ed34dbba91712384518f6f2" candidate_uuid: "2ee1436310174f20aa6e60f0e644cc93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d40021626873405fb8183692867ac93f"
I20260809 15:10:57.195358 17970 raft_consensus.cc:3037] T f66eedcf6ed34dbba91712384518f6f2 P e6fbb915997b4f3da1537556271642f6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:10:57.195453 18415 raft_consensus.cc:3037] T f66eedcf6ed34dbba91712384518f6f2 P d40021626873405fb8183692867ac93f [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:10:57.197830 17970 raft_consensus.cc:2454] T f66eedcf6ed34dbba91712384518f6f2 P e6fbb915997b4f3da1537556271642f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ee1436310174f20aa6e60f0e644cc93 in term 1.
I20260809 15:10:57.197867 18415 raft_consensus.cc:2454] T f66eedcf6ed34dbba91712384518f6f2 P d40021626873405fb8183692867ac93f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ee1436310174f20aa6e60f0e644cc93 in term 1.
I20260809 15:10:57.198235 18076 leader_election.cc:302] T f66eedcf6ed34dbba91712384518f6f2 P 2ee1436310174f20aa6e60f0e644cc93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ee1436310174f20aa6e60f0e644cc93, e6fbb915997b4f3da1537556271642f6; no voters:
I20260809 15:10:57.198505 18530 raft_consensus.cc:2781] T f66eedcf6ed34dbba91712384518f6f2 P 2ee1436310174f20aa6e60f0e644cc93 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:10:57.198958 18530 raft_consensus.cc:686] T f66eedcf6ed34dbba91712384518f6f2 P 2ee1436310174f20aa6e60f0e644cc93 [term 1 LEADER]: Becoming Leader. State: Replica: 2ee1436310174f20aa6e60f0e644cc93, State: Running, Role: LEADER
I20260809 15:10:57.199491 18530 consensus_queue.cc:237] T f66eedcf6ed34dbba91712384518f6f2 P 2ee1436310174f20aa6e60f0e644cc93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6fbb915997b4f3da1537556271642f6" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 46463 } } peers { permanent_uuid: "2ee1436310174f20aa6e60f0e644cc93" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 42299 } } peers { permanent_uuid: "d40021626873405fb8183692867ac93f" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38479 } }
I20260809 15:10:57.204079 17714 catalog_manager.cc:5495] T f66eedcf6ed34dbba91712384518f6f2 P 2ee1436310174f20aa6e60f0e644cc93 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ee1436310174f20aa6e60f0e644cc93 (127.24.183.194). New cstate: current_term: 1 leader_uuid: "2ee1436310174f20aa6e60f0e644cc93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6fbb915997b4f3da1537556271642f6" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 46463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ee1436310174f20aa6e60f0e644cc93" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 42299 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d40021626873405fb8183692867ac93f" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38479 } health_report { overall_health: UNKNOWN } } }
W20260809 15:10:57.213286 18495 tablet.cc:2369] T f66eedcf6ed34dbba91712384518f6f2 P d40021626873405fb8183692867ac93f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:10:57.233304 18517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:57.227561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53902 (local address 127.24.183.194:42299)
0809 15:10:57.227804 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:57.227811 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:57.227834 (+ 23us) server_negotiation.cc:407] Connection header received
0809 15:10:57.227961 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:57.227965 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:57.228067 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:57.228224 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:10:57.228642 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:57.229395 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:57.231095 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:57.231541 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:57.231662 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:57.232181 (+ 519us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:10:57.232680 (+ 499us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:10:57.232828 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:57.232939 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 15:10:57.233024 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 15:10:57.241847 18415 raft_consensus.cc:1261] T f66eedcf6ed34dbba91712384518f6f2 P d40021626873405fb8183692867ac93f [term 1 FOLLOWER]: Refusing update from remote peer 2ee1436310174f20aa6e60f0e644cc93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:10:57.241835 17970 raft_consensus.cc:1261] T f66eedcf6ed34dbba91712384518f6f2 P e6fbb915997b4f3da1537556271642f6 [term 1 FOLLOWER]: Refusing update from remote peer 2ee1436310174f20aa6e60f0e644cc93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:10:57.242789 18530 consensus_queue.cc:1035] T f66eedcf6ed34dbba91712384518f6f2 P 2ee1436310174f20aa6e60f0e644cc93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6fbb915997b4f3da1537556271642f6" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 46463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:10:57.243103 18537 consensus_queue.cc:1035] T f66eedcf6ed34dbba91712384518f6f2 P 2ee1436310174f20aa6e60f0e644cc93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d40021626873405fb8183692867ac93f" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:10:57.254902 18546 mvcc.cc:205] Tried to move back new op lower bound from 7316636701651664896 to 7316636701493522432. Current Snapshot: MvccSnapshot[applied={T|T < 7316636701651664896}]
I20260809 15:10:57.257294 18549 mvcc.cc:205] Tried to move back new op lower bound from 7316636701651664896 to 7316636701493522432. Current Snapshot: MvccSnapshot[applied={T|T < 7316636701651664896}]
I20260809 15:10:57.270498 17717 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:10:57.270783 17717 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:10:57.274645 17717 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50: Sending DeleteTablet for 3 replicas of tablet f66eedcf6ed34dbba91712384518f6f2
I20260809 15:10:57.275846 17935 tablet_service.cc:1514] Processing DeleteTablet for tablet f66eedcf6ed34dbba91712384518f6f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:10:57 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:44692
I20260809 15:10:57.275852 18158 tablet_service.cc:1514] Processing DeleteTablet for tablet f66eedcf6ed34dbba91712384518f6f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:10:57 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:53896
I20260809 15:10:57.276772 18385 tablet_service.cc:1514] Processing DeleteTablet for tablet f66eedcf6ed34dbba91712384518f6f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:10:57 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:54788
I20260809 15:10:57.277678 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 17829
I20260809 15:10:57.287950 18566 tablet_replica.cc:330] T f66eedcf6ed34dbba91712384518f6f2 P 2ee1436310174f20aa6e60f0e644cc93: stopping tablet replica
I20260809 15:10:57.288436 18566 raft_consensus.cc:2229] T f66eedcf6ed34dbba91712384518f6f2 P 2ee1436310174f20aa6e60f0e644cc93 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:10:57.289327 18568 tablet_replica.cc:330] T f66eedcf6ed34dbba91712384518f6f2 P d40021626873405fb8183692867ac93f: stopping tablet replica
I20260809 15:10:57.289778 18568 raft_consensus.cc:2229] T f66eedcf6ed34dbba91712384518f6f2 P d40021626873405fb8183692867ac93f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:10:57.290364 18568 raft_consensus.cc:2258] T f66eedcf6ed34dbba91712384518f6f2 P d40021626873405fb8183692867ac93f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:10:57.291272 18566 raft_consensus.cc:2258] T f66eedcf6ed34dbba91712384518f6f2 P 2ee1436310174f20aa6e60f0e644cc93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:10:57.293083 18568 ts_tablet_manager.cc:1905] T f66eedcf6ed34dbba91712384518f6f2 P d40021626873405fb8183692867ac93f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:10:57.296487 18567 tablet_replica.cc:330] T f66eedcf6ed34dbba91712384518f6f2 P e6fbb915997b4f3da1537556271642f6: stopping tablet replica
I20260809 15:10:57.296885 18567 raft_consensus.cc:2229] T f66eedcf6ed34dbba91712384518f6f2 P e6fbb915997b4f3da1537556271642f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:10:57.297359 18567 raft_consensus.cc:2258] T f66eedcf6ed34dbba91712384518f6f2 P e6fbb915997b4f3da1537556271642f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:10:57.300292 18567 ts_tablet_manager.cc:1905] T f66eedcf6ed34dbba91712384518f6f2 P e6fbb915997b4f3da1537556271642f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:10:57.302080 18566 ts_tablet_manager.cc:1905] T f66eedcf6ed34dbba91712384518f6f2 P 2ee1436310174f20aa6e60f0e644cc93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:10:57.302398 18568 ts_tablet_manager.cc:1918] T f66eedcf6ed34dbba91712384518f6f2 P d40021626873405fb8183692867ac93f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:10:57.302556 18568 log.cc:1196] T f66eedcf6ed34dbba91712384518f6f2 P d40021626873405fb8183692867ac93f: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/wal/wals/f66eedcf6ed34dbba91712384518f6f2
I20260809 15:10:57.303068 18568 ts_tablet_manager.cc:1939] T f66eedcf6ed34dbba91712384518f6f2 P d40021626873405fb8183692867ac93f: Deleting consensus metadata
I20260809 15:10:57.305924 17689 catalog_manager.cc:4841] TS d40021626873405fb8183692867ac93f (127.24.183.195:38479): tablet f66eedcf6ed34dbba91712384518f6f2 (table test-table [id=48624c77f2aa4a45b073dfea786d2584]) successfully deleted
I20260809 15:10:57.307926 18567 ts_tablet_manager.cc:1918] T f66eedcf6ed34dbba91712384518f6f2 P e6fbb915997b4f3da1537556271642f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:10:57.308135 18567 log.cc:1196] T f66eedcf6ed34dbba91712384518f6f2 P e6fbb915997b4f3da1537556271642f6: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/wal/wals/f66eedcf6ed34dbba91712384518f6f2
I20260809 15:10:57.308723 18567 ts_tablet_manager.cc:1939] T f66eedcf6ed34dbba91712384518f6f2 P e6fbb915997b4f3da1537556271642f6: Deleting consensus metadata
I20260809 15:10:57.309463 18566 ts_tablet_manager.cc:1918] T f66eedcf6ed34dbba91712384518f6f2 P 2ee1436310174f20aa6e60f0e644cc93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:10:57.309649 18566 log.cc:1196] T f66eedcf6ed34dbba91712384518f6f2 P 2ee1436310174f20aa6e60f0e644cc93: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/wal/wals/f66eedcf6ed34dbba91712384518f6f2
I20260809 15:10:57.310185 18566 ts_tablet_manager.cc:1939] T f66eedcf6ed34dbba91712384518f6f2 P 2ee1436310174f20aa6e60f0e644cc93: Deleting consensus metadata
Aug 09 15:10:57 dist-test-slave-nj21 krb5kdc[17634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.193@KRBTEST.COM
I20260809 15:10:57.310691 17689 catalog_manager.cc:4841] TS e6fbb915997b4f3da1537556271642f6 (127.24.183.193:46463): tablet f66eedcf6ed34dbba91712384518f6f2 (table test-table [id=48624c77f2aa4a45b073dfea786d2584]) successfully deleted
I20260809 15:10:57.312398 17688 catalog_manager.cc:4841] TS 2ee1436310174f20aa6e60f0e644cc93 (127.24.183.194:42299): tablet f66eedcf6ed34dbba91712384518f6f2 (table test-table [id=48624c77f2aa4a45b073dfea786d2584]) successfully deleted
I20260809 15:10:57.317271 18514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:57.278811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44698 (local address 127.24.183.193:46463)
0809 15:10:57.278986 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:57.278993 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:57.286323 (+ 7330us) server_negotiation.cc:407] Connection header received
0809 15:10:57.286507 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:57.286513 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:57.286606 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:57.286710 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:57.291396 (+ 4686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:57.292904 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:57.294994 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:57.300605 (+ 5611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:57.300716 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:57.311703 (+ 10987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:57.311718 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:57.311736 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:57.311776 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:57.314459 (+ 2683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:57.315183 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:57.315190 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:57.315192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:57.315332 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:57.315741 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:57.315746 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:57.315749 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:57.316079 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:57.316373 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:57.316709 (+ 336us) server_negotiation.cc:299] Negotiation successful
0809 15:10:57.316922 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 15:10:57.317898 18033 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:10:57.407631 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 17829
I20260809 15:10:57.437731 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 18050
Aug 09 15:10:57 dist-test-slave-nj21 krb5kdc[17634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.194@KRBTEST.COM
I20260809 15:10:57.457872 18517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:57.438528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53906 (local address 127.24.183.194:42299)
0809 15:10:57.438729 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:57.438735 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:57.438915 (+ 180us) server_negotiation.cc:407] Connection header received
0809 15:10:57.439125 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:57.439130 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:57.439216 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:57.439346 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:57.441171 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:57.442376 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:57.444264 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:57.444916 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:57.445035 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:57.450509 (+ 5474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:57.450523 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:57.450541 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:57.450589 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:57.454820 (+ 4231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:57.455528 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:57.455536 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:57.455540 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:57.455727 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:57.456159 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:57.456165 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:57.456168 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:57.456641 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:57.456966 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:57.457235 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 15:10:57.457487 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 15:10:57.458768 18259 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:10:57.574841 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 18050
I20260809 15:10:57.599417 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 18279
Aug 09 15:10:57 dist-test-slave-nj21 krb5kdc[17634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.195@KRBTEST.COM
I20260809 15:10:57.614576 18521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:10:57.600155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54798 (local address 127.24.183.195:38479)
0809 15:10:57.600361 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 15:10:57.600367 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:10:57.600430 (+ 63us) server_negotiation.cc:407] Connection header received
0809 15:10:57.600594 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:10:57.600599 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:10:57.600667 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:10:57.600750 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:10:57.602305 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:57.603031 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:57.604463 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:10:57.604851 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:10:57.604942 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:10:57.609126 (+ 4184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:10:57.609140 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:10:57.609157 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:10:57.609194 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:10:57.612243 (+ 3049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:57.612920 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:57.612926 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:57.612928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:57.613050 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:10:57.613318 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:10:57.613321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:10:57.613323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:10:57.613653 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:10:57.613891 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 15:10:57.614086 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 15:10:57.614247 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 15:10:57.615204 18489 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:10:57.706269 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 18279
I20260809 15:10:57.728722 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 17653
I20260809 15:10:57.729964 17776 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:10:57.841598 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 17653
I20260809 15:11:17.871654 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:35881
--webserver_interface=127.24.183.254
--webserver_port=36299
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:35177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:35881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:11:18.224695 18577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:11:18.224982 18577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:11:18.225075 18577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:11:18.232100 18577 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 15:11:18.232187 18577 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 15:11:18.232224 18577 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 15:11:18.232275 18577 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:11:18.232327 18577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:11:18.232394 18577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:11:18.232439 18577 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:11:18.232492 18577 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:11:18.241991 18577 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:35177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:35881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:35881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=36299
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:11:18.243176 18577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:11:18.244498 18577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:11:18.268842 18585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:11:18.268875 18588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:11:18.268801 18586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:11:18.269142 18577 server_base.cc:1034] running on GCE node
I20260809 15:11:18.270265 18577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:11:18.271829 18577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:11:18.273061 18577 hybrid_clock.cc:648] HybridClock initialized: now 1786288278272982 us; error 44 us; skew 500 ppm
I20260809 15:11:18.273454 18577 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:11:18 dist-test-slave-nj21 krb5kdc[17634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288278, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:11:18.276957 18577 init.cc:377] Logged in from keytab as kudu/127.24.183.254@KRBTEST.COM (short username kudu)
I20260809 15:11:18.278529 18577 webserver.cc:460] Webserver started at http://127.24.183.254:36299/ using document root <none> and password file <none>
I20260809 15:11:18.279274 18577 fs_manager.cc:359] Metadata directory not provided
I20260809 15:11:18.279397 18577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:11:18.285244 18577 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260809 15:11:18.288249 18597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:11:18.289693 18577 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:11:18.289857 18577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "e1f1911f3a9b41b88d359babf89cda50"
format_stamp: "Formatted at 2026-08-09 15:10:55 on dist-test-slave-nj21"
server_key: "84af479603da0ec7563a902ff48c2210"
server_key_iv: "3412991e4a96584bea80206364fe9bd3"
server_key_version: "encryptionkey@0"
I20260809 15:11:18.290881 18577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:11:18.309073 18577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:11:18.313169 18577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:11:18.313449 18577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:11:18.332084 18577 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:35881
I20260809 15:11:18.332116 18683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:35881 every 8 connection(s)
I20260809 15:11:18.333643 18577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:11:18.340822 18685 sys_catalog.cc:263] Verifying existing consensus state
I20260809 15:11:18.342060 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 18577
I20260809 15:11:18.343086 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.193:46463
--local_ip_for_outbound_sockets=127.24.183.193
--tserver_master_addrs=127.24.183.254:35881
--webserver_port=41175
--webserver_interface=127.24.183.193
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:35177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:11:18.345412 18685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50: Bootstrap starting.
I20260809 15:11:18.371592 18685 log.cc:824] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50: Log is configured to *not* fsync() on all Append() calls
I20260809 15:11:18.388581 18685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:11:18.389317 18685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50: Bootstrap complete.
I20260809 15:11:18.403087 18685 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f1911f3a9b41b88d359babf89cda50" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 35881 } }
I20260809 15:11:18.404320 18685 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f1911f3a9b41b88d359babf89cda50, State: Initialized, Role: FOLLOWER
I20260809 15:11:18.405071 18685 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f1911f3a9b41b88d359babf89cda50" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 35881 } }
I20260809 15:11:18.405370 18685 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:11:18.405499 18685 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:11:18.405671 18685 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:11:18.411494 18685 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f1911f3a9b41b88d359babf89cda50" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 35881 } }
I20260809 15:11:18.412168 18685 leader_election.cc:302] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1f1911f3a9b41b88d359babf89cda50; no voters:
I20260809 15:11:18.412881 18685 leader_election.cc:288] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 15:11:18.413062 18691 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:11:18.414157 18691 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [term 2 LEADER]: Becoming Leader. State: Replica: e1f1911f3a9b41b88d359babf89cda50, State: Running, Role: LEADER
I20260809 15:11:18.414667 18691 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f1911f3a9b41b88d359babf89cda50" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 35881 } }
I20260809 15:11:18.415252 18685 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:11:18.417233 18696 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1f1911f3a9b41b88d359babf89cda50. Latest consensus state: current_term: 2 leader_uuid: "e1f1911f3a9b41b88d359babf89cda50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f1911f3a9b41b88d359babf89cda50" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 35881 } } }
I20260809 15:11:18.417153 18694 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e1f1911f3a9b41b88d359babf89cda50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f1911f3a9b41b88d359babf89cda50" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 35881 } } }
I20260809 15:11:18.417522 18696 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [sys.catalog]: This master's current role is: LEADER
I20260809 15:11:18.417518 18694 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [sys.catalog]: This master's current role is: LEADER
I20260809 15:11:18.431495 18702 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:11:18.439605 18702 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:11:18.441046 18702 catalog_manager.cc:1259] Loaded cluster ID: 9ff381450f124676b3c29c4fed0a7ac4
I20260809 15:11:18.441151 18702 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:11:18.445180 18702 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:11:18.446769 18702 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50: Loaded TSK: 0
I20260809 15:11:18.447410 18722 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 15:11:18.476584 18702 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50: Generated new TSK 1
I20260809 15:11:18.477038 18702 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 15:11:18.716261 18688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:11:18.716552 18688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:11:18.716689 18688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:11:18.724097 18688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:11:18.724218 18688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:11:18.724371 18688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.193
I20260809 15:11:18.734237 18688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:35177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.193:46463
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.183.193
--webserver_port=41175
--tserver_master_addrs=127.24.183.254:35881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.193
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:11:18.735582 18688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:11:18.736937 18688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:11:18.751420 18733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:11:18.751394 18734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:11:18.752220 18736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:11:18.753278 18688 server_base.cc:1034] running on GCE node
I20260809 15:11:18.753862 18688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:11:18.754796 18688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:11:18.756009 18688 hybrid_clock.cc:648] HybridClock initialized: now 1786288278755954 us; error 47 us; skew 500 ppm
I20260809 15:11:18.756307 18688 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:11:18 dist-test-slave-nj21 krb5kdc[17634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288278, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:11:18.759446 18688 init.cc:377] Logged in from keytab as kudu/127.24.183.193@KRBTEST.COM (short username kudu)
I20260809 15:11:18.760953 18688 webserver.cc:460] Webserver started at http://127.24.183.193:41175/ using document root <none> and password file <none>
I20260809 15:11:18.761624 18688 fs_manager.cc:359] Metadata directory not provided
I20260809 15:11:18.761750 18688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:11:18.766978 18688 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260809 15:11:18.769647 18747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:11:18.770952 18688 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:11:18.771104 18688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/wal
uuid: "e6fbb915997b4f3da1537556271642f6"
format_stamp: "Formatted at 2026-08-09 15:10:55 on dist-test-slave-nj21"
server_key: "12c0b4701e3adf4c3c503c274d80b43a"
server_key_iv: "44e535ee8626ae52197aabd6f2d29f1d"
server_key_version: "encryptionkey@0"
I20260809 15:11:18.772130 18688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:11:18.786427 18688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:11:18.790743 18688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:11:18.791015 18688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:11:18.792101 18688 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:11:18.793541 18688 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:11:18.793632 18688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:11:18.793741 18688 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:11:18.793808 18688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:11:18.825508 18688 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.193:46463
I20260809 15:11:18.825579 18922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.193:46463 every 8 connection(s)
I20260809 15:11:18.827013 18688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
I20260809 15:11:18.830374 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 18688
I20260809 15:11:18.831319 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.194:42299
--local_ip_for_outbound_sockets=127.24.183.194
--tserver_master_addrs=127.24.183.254:35881
--webserver_port=36033
--webserver_interface=127.24.183.194
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:35177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 15:11:18 dist-test-slave-nj21 krb5kdc[17634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288278, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:11:18.852684 18932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:18.830178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:44891 (local address 127.24.183.254:35881)
0809 15:11:18.830941 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0809 15:11:18.830958 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 15:11:18.834399 (+ 3441us) server_negotiation.cc:407] Connection header received
0809 15:11:18.834755 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:11:18.834772 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:11:18.835201 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:11:18.835590 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:11:18.837103 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:18.838906 (+ 1803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:18.840514 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:18.841427 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:18.841601 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:11:18.845262 (+ 3661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:11:18.845269 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:11:18.845283 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:11:18.845326 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:11:18.848316 (+ 2990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:11:18.849152 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:11:18.849159 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:11:18.849164 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:11:18.849342 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:11:18.849825 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:11:18.849828 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:11:18.849830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:11:18.850290 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:11:18.850453 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 15:11:18.851354 (+ 901us) server_negotiation.cc:299] Negotiation successful
0809 15:11:18.851703 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":297,"threads_started":1}
I20260809 15:11:18.852890 18933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:18.830257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.193:44891)
0809 15:11:18.831591 (+ 1334us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:18.831652 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 15:11:18.834008 (+ 2356us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:11:18.835815 (+ 1807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:11:18.835823 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:11:18.836345 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:11:18.836891 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:11:18.836910 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:11:18.839074 (+ 2164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:11:18.839078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:11:18.840376 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:11:18.840389 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:11:18.841641 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:11:18.841646 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:11:18.841857 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:11:18.842490 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:11:18.842518 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:11:18.844963 (+ 2445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:11:18.848454 (+ 3491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:11:18.848462 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:11:18.848475 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:11:18.849014 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:11:18.849466 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:11:18.849472 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:11:18.849475 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:11:18.849718 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:11:18.850429 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:11:18.850436 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:11:18.850889 (+ 453us) client_negotiation.cc:769] Sending connection context
0809 15:11:18.851162 (+ 273us) client_negotiation.cc:240] Negotiation successful
0809 15:11:18.851702 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":782,"thread_start_us":515,"threads_started":1}
I20260809 15:11:18.855531 18923 heartbeater.cc:347] Connected to a master server at 127.24.183.254:35881
I20260809 15:11:18.855870 18923 heartbeater.cc:464] Registering TS with master...
I20260809 15:11:18.856613 18923 heartbeater.cc:511] Master 127.24.183.254:35881 requested a full tablet report, sending...
I20260809 15:11:18.858744 18620 ts_manager.cc:194] Registered new tserver with Master: e6fbb915997b4f3da1537556271642f6 (127.24.183.193:46463)
I20260809 15:11:18.860289 18620 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.193@KRBTEST.COM'} at 127.24.183.193:44891
W20260809 15:11:19.174109 18934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:11:19.174396 18934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:11:19.174532 18934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:11:19.181689 18934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:11:19.181813 18934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:11:19.181969 18934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.194
I20260809 15:11:19.191581 18934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:35177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.194:42299
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.183.194
--webserver_port=36033
--tserver_master_addrs=127.24.183.254:35881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.194
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:11:19.192875 18934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:11:19.194162 18934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:11:19.207351 18948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:11:19.207419 18940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:11:19.207391 18950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:11:19.208160 18934 server_base.cc:1034] running on GCE node
I20260809 15:11:19.208734 18934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:11:19.209635 18934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:11:19.210843 18934 hybrid_clock.cc:648] HybridClock initialized: now 1786288279210787 us; error 47 us; skew 500 ppm
I20260809 15:11:19.211141 18934 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:11:19 dist-test-slave-nj21 krb5kdc[17634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288279, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:11:19.214176 18934 init.cc:377] Logged in from keytab as kudu/127.24.183.194@KRBTEST.COM (short username kudu)
I20260809 15:11:19.215591 18934 webserver.cc:460] Webserver started at http://127.24.183.194:36033/ using document root <none> and password file <none>
I20260809 15:11:19.216296 18934 fs_manager.cc:359] Metadata directory not provided
I20260809 15:11:19.216431 18934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:11:19.221747 18934 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.004s
I20260809 15:11:19.224500 18961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:11:19.225780 18934 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:11:19.225930 18934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/wal
uuid: "2ee1436310174f20aa6e60f0e644cc93"
format_stamp: "Formatted at 2026-08-09 15:10:56 on dist-test-slave-nj21"
server_key: "5b45486b0132c29cdb81515d61c07361"
server_key_iv: "86e6e5440eda4ded60d4be036963a159"
server_key_version: "encryptionkey@0"
I20260809 15:11:19.226881 18934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:11:19.244879 18934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:11:19.248947 18934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:11:19.249209 18934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:11:19.250208 18934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:11:19.251572 18934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:11:19.251713 18934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:11:19.251832 18934 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:11:19.251909 18934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:11:19.284317 18934 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.194:42299
I20260809 15:11:19.284403 19148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.194:42299 every 8 connection(s)
I20260809 15:11:19.285746 18934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
I20260809 15:11:19.293424 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 18934
I20260809 15:11:19.294080 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.195:38479
--local_ip_for_outbound_sockets=127.24.183.195
--tserver_master_addrs=127.24.183.254:35881
--webserver_port=44075
--webserver_interface=127.24.183.195
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:35177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 15:11:19 dist-test-slave-nj21 krb5kdc[17634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288279, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:11:19.313449 18932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:19.288332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:43477 (local address 127.24.183.254:35881)
0809 15:11:19.288510 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 15:11:19.288514 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:11:19.289720 (+ 1206us) server_negotiation.cc:407] Connection header received
0809 15:11:19.291632 (+ 1912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:11:19.291638 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:11:19.291756 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:11:19.291851 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:11:19.293575 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:19.294386 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:19.296631 (+ 2245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:19.297066 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:19.297162 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:11:19.301160 (+ 3998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:11:19.301165 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:11:19.301168 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:11:19.301203 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:11:19.303653 (+ 2450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:11:19.304521 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:11:19.304527 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:11:19.304530 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:11:19.304778 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:11:19.311415 (+ 6637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:11:19.311421 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:11:19.311424 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:11:19.311856 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:11:19.312004 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 15:11:19.312853 (+ 849us) server_negotiation.cc:299] Negotiation successful
0809 15:11:19.313096 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 15:11:19.314530 19156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:19.288763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.194:43477)
0809 15:11:19.289558 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:19.289598 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 15:11:19.291346 (+ 1748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:11:19.292125 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:11:19.292133 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:11:19.292643 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:11:19.293366 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:11:19.293394 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:11:19.294566 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:11:19.294570 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:11:19.296470 (+ 1900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:11:19.296486 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:11:19.297289 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:11:19.297296 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:11:19.297579 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:11:19.298256 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:11:19.298283 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:11:19.300926 (+ 2643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:11:19.303817 (+ 2891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:11:19.303825 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:11:19.303838 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:11:19.304364 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:11:19.304918 (+ 554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:11:19.304924 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:11:19.304927 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:11:19.305271 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:11:19.311986 (+ 6715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:11:19.311994 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:11:19.312430 (+ 436us) client_negotiation.cc:769] Sending connection context
0809 15:11:19.312715 (+ 285us) client_negotiation.cc:240] Negotiation successful
0809 15:11:19.313239 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":334,"threads_started":1}
I20260809 15:11:19.316150 19149 heartbeater.cc:347] Connected to a master server at 127.24.183.254:35881
I20260809 15:11:19.316476 19149 heartbeater.cc:464] Registering TS with master...
I20260809 15:11:19.317214 19149 heartbeater.cc:511] Master 127.24.183.254:35881 requested a full tablet report, sending...
I20260809 15:11:19.318835 18620 ts_manager.cc:194] Registered new tserver with Master: 2ee1436310174f20aa6e60f0e644cc93 (127.24.183.194:42299)
I20260809 15:11:19.319777 18620 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.194@KRBTEST.COM'} at 127.24.183.194:43477
W20260809 15:11:19.640040 19157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:11:19.640307 19157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:11:19.640435 19157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:11:19.647423 19157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:11:19.647519 19157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:11:19.647665 19157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.195
I20260809 15:11:19.657008 19157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:35177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.195:38479
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.183.195
--webserver_port=44075
--tserver_master_addrs=127.24.183.254:35881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.195
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:11:19.658171 19157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:11:19.659464 19157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:11:19.671885 19167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:11:19.671931 19171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:11:19.671972 19168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:11:19.673363 19157 server_base.cc:1034] running on GCE node
I20260809 15:11:19.673941 19157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:11:19.674840 19157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:11:19.676082 19157 hybrid_clock.cc:648] HybridClock initialized: now 1786288279676027 us; error 37 us; skew 500 ppm
I20260809 15:11:19.676465 19157 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:11:19 dist-test-slave-nj21 krb5kdc[17634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288279, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:11:19.680554 19157 init.cc:377] Logged in from keytab as kudu/127.24.183.195@KRBTEST.COM (short username kudu)
I20260809 15:11:19.682686 19157 webserver.cc:460] Webserver started at http://127.24.183.195:44075/ using document root <none> and password file <none>
I20260809 15:11:19.683382 19157 fs_manager.cc:359] Metadata directory not provided
I20260809 15:11:19.683524 19157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:11:19.689184 19157 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.001s
I20260809 15:11:19.691944 19182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:11:19.693295 19157 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:11:19.693447 19157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/wal
uuid: "d40021626873405fb8183692867ac93f"
format_stamp: "Formatted at 2026-08-09 15:10:56 on dist-test-slave-nj21"
server_key: "ea035a8094f632db2b52fe36227f7a2e"
server_key_iv: "fc5815666a05ebee49041246e29646ae"
server_key_version: "encryptionkey@0"
I20260809 15:11:19.694437 19157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:11:19.732265 19157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:11:19.736377 19157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:11:19.736642 19157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:11:19.737648 19157 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:11:19.738958 19157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:11:19.739055 19157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:11:19.739161 19157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:11:19.739231 19157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:11:19.771502 19157 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.195:38479
I20260809 15:11:19.771552 19367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.195:38479 every 8 connection(s)
I20260809 15:11:19.773005 19157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
I20260809 15:11:19.779345 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 19157
Aug 09 15:11:19 dist-test-slave-nj21 krb5kdc[17634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288279, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:11:19.795172 18932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:19.775634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:39773 (local address 127.24.183.254:35881)
0809 15:11:19.775872 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 15:11:19.775879 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:11:19.777010 (+ 1131us) server_negotiation.cc:407] Connection header received
0809 15:11:19.778846 (+ 1836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:11:19.778853 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:11:19.778948 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:11:19.779061 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:11:19.780904 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:19.781957 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:19.783748 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:19.784258 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:19.784380 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:11:19.788441 (+ 4061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:11:19.788446 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:11:19.788449 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:11:19.788478 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:11:19.791993 (+ 3515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:11:19.792719 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:11:19.792725 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:11:19.792728 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:11:19.792870 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:11:19.793345 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:11:19.793348 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:11:19.793350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:11:19.793687 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:11:19.793821 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 15:11:19.794544 (+ 723us) server_negotiation.cc:299] Negotiation successful
0809 15:11:19.794724 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260809 15:11:19.795811 19373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:19.780261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41170 (local address 127.24.183.254:35881)
0809 15:11:19.780760 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0809 15:11:19.780766 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:11:19.781222 (+ 456us) server_negotiation.cc:407] Connection header received
0809 15:11:19.781370 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:11:19.781374 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:11:19.781434 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:11:19.781529 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:11:19.783203 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:19.784171 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:19.785734 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:19.786368 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:19.786505 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:11:19.789549 (+ 3044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:11:19.789556 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:11:19.789560 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:11:19.789610 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:11:19.792650 (+ 3040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:11:19.793416 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:11:19.793420 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:11:19.793423 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:11:19.793534 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:11:19.793943 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:11:19.793946 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:11:19.793948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:11:19.794191 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:11:19.794382 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 15:11:19.794919 (+ 537us) server_negotiation.cc:299] Negotiation successful
0809 15:11:19.795117 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":258,"threads_started":1}
I20260809 15:11:19.796295 19371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:19.776061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.195:39773)
0809 15:11:19.776845 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:19.776896 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 15:11:19.778563 (+ 1667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:11:19.779271 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:11:19.779279 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:11:19.779874 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:11:19.780700 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:11:19.780729 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:11:19.782128 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:11:19.782132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:11:19.783570 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:11:19.783586 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:11:19.784464 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:11:19.784469 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:11:19.784662 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:11:19.785366 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:11:19.785389 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:11:19.788230 (+ 2841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:11:19.792127 (+ 3897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:11:19.792134 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:11:19.792145 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:11:19.792591 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:11:19.792982 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:11:19.792987 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:11:19.792990 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:11:19.793231 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:11:19.793812 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:11:19.793820 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:11:19.794161 (+ 341us) client_negotiation.cc:769] Sending connection context
0809 15:11:19.794436 (+ 275us) client_negotiation.cc:240] Negotiation successful
0809 15:11:19.794952 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":277,"threads_started":1}
I20260809 15:11:19.797616 19369 heartbeater.cc:347] Connected to a master server at 127.24.183.254:35881
I20260809 15:11:19.797920 19369 heartbeater.cc:464] Registering TS with master...
I20260809 15:11:19.798616 19369 heartbeater.cc:511] Master 127.24.183.254:35881 requested a full tablet report, sending...
I20260809 15:11:19.800007 18620 ts_manager.cc:194] Registered new tserver with Master: d40021626873405fb8183692867ac93f (127.24.183.195:38479)
I20260809 15:11:19.800791 18620 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.195@KRBTEST.COM'} at 127.24.183.195:39773
I20260809 15:11:19.810367 25311 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:11:19.816731 19373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:19.811257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41182 (local address 127.24.183.254:35881)
0809 15:11:19.811426 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 15:11:19.811430 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:11:19.811858 (+ 428us) server_negotiation.cc:407] Connection header received
0809 15:11:19.812016 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:11:19.812020 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:11:19.812162 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:11:19.812289 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:11:19.812644 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:19.813346 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:19.814857 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:19.815346 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:19.815433 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:11:19.815919 (+ 486us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:11:19.816199 (+ 280us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 15:11:19.816498 (+ 299us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41182: token expired
Metrics: {"server-negotiator.queue_time_us":48}
W20260809 15:11:19.816831 19373 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41182: token expired
W20260809 15:11:19.817325 18498 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.24.183.254:35881)
I20260809 15:11:19.830242 19373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:19.817809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41188 (local address 127.24.183.254:35881)
0809 15:11:19.817966 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 15:11:19.817971 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:11:19.817988 (+ 17us) server_negotiation.cc:407] Connection header received
0809 15:11:19.818102 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:11:19.818106 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:11:19.818160 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:11:19.818234 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:11:19.819590 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:19.820292 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:19.821600 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:19.821988 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:19.822097 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:11:19.824626 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:11:19.824631 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:11:19.824634 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:11:19.824664 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:11:19.827730 (+ 3066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:11:19.828352 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:11:19.828357 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:11:19.828360 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:11:19.828506 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:11:19.828831 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:11:19.828835 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:11:19.828838 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:11:19.829233 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:11:19.829480 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0809 15:11:19.829706 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 15:11:19.829951 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 15:11:19.833567 18620 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41188:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:11:19.840847 18620 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:11:19.863287 18923 heartbeater.cc:503] Master 127.24.183.254:35881 was elected leader, sending a full tablet report...
I20260809 15:11:19.863615 19396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:19.854103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:46463 (local address 127.0.0.1:57314)
0809 15:11:19.856126 (+ 2023us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:19.856143 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:11:19.856268 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:11:19.856881 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:11:19.856888 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:11:19.856913 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:11:19.857236 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:11:19.857246 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:11:19.858549 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:11:19.858555 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:11:19.861210 (+ 2655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:11:19.861227 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:11:19.862606 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:11:19.862613 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:11:19.862830 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:11:19.862874 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 15:11:19.862980 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 15:11:19.863156 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1918,"mutex_wait_us":53,"thread_start_us":1771,"threads_started":1}
I20260809 15:11:19.864075 19388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:19.852066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:38479 (local address 127.0.0.1:35588)
0809 15:11:19.855281 (+ 3215us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:19.855298 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:11:19.855489 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:11:19.856342 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:11:19.856349 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:11:19.856378 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:11:19.856819 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:11:19.856833 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:11:19.858899 (+ 2066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:11:19.858904 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:11:19.861091 (+ 2187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:11:19.861107 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:11:19.863036 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:11:19.863043 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:11:19.863208 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:11:19.863255 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 15:11:19.863343 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 15:11:19.863481 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3098,"mutex_wait_us":159,"thread_start_us":1250,"threads_started":1}
I20260809 15:11:19.864733 19398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:19.854737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57314 (local address 127.24.183.193:46463)
0809 15:11:19.855688 (+ 951us) server_negotiation.cc:206] Beginning negotiation
0809 15:11:19.855697 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:11:19.856392 (+ 695us) server_negotiation.cc:407] Connection header received
0809 15:11:19.856502 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:11:19.856509 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:11:19.856700 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:11:19.856867 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:11:19.857378 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:19.858419 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:19.861362 (+ 2943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:19.862397 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:19.862582 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:11:19.862757 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 15:11:19.863056 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 15:11:19.864415 (+ 1359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":763,"thread_start_us":622,"threads_started":1}
I20260809 15:11:19.865160 19394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:19.852657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35588 (local address 127.24.183.195:38479)
0809 15:11:19.854542 (+ 1885us) server_negotiation.cc:206] Beginning negotiation
0809 15:11:19.854551 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:11:19.855386 (+ 835us) server_negotiation.cc:407] Connection header received
0809 15:11:19.855703 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:11:19.855711 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:11:19.855960 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:11:19.856119 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:11:19.857064 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:19.858759 (+ 1695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:19.861257 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:19.862514 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:19.862703 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:11:19.862867 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 15:11:19.863734 (+ 867us) server_negotiation.cc:299] Negotiation successful
0809 15:11:19.863897 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1717,"thread_start_us":1560,"threads_started":1}
I20260809 15:11:19.865767 19389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:19.851921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37996 (local address 127.24.183.194:42299)
0809 15:11:19.854957 (+ 3036us) server_negotiation.cc:206] Beginning negotiation
0809 15:11:19.854966 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:11:19.854994 (+ 28us) server_negotiation.cc:407] Connection header received
0809 15:11:19.855105 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:11:19.855113 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:11:19.856271 (+ 1158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:11:19.856470 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:11:19.857086 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:19.858722 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:19.861694 (+ 2972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:19.863141 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:19.865094 (+ 1953us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:11:19.865217 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 15:11:19.865297 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 15:11:19.865453 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2903,"thread_start_us":2769,"threads_started":1}
I20260809 15:11:19.866914 19393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:19.851619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:42299 (local address 127.0.0.1:37996)
0809 15:11:19.853720 (+ 2101us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:19.853766 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 15:11:19.853922 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:11:19.856464 (+ 2542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:11:19.856468 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:11:19.856495 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:11:19.856899 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:11:19.856908 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:11:19.858900 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:11:19.858904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:11:19.861532 (+ 2628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:11:19.861546 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:11:19.863401 (+ 1855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:11:19.863408 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:11:19.863571 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:11:19.863612 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 15:11:19.864451 (+ 839us) client_negotiation.cc:240] Negotiation successful
0809 15:11:19.864574 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1850,"mutex_wait_us":148,"thread_start_us":3408,"threads_started":1}
I20260809 15:11:19.868672 18821 tablet_service.cc:1467] Processing CreateTablet for tablet 2e8a690421f44a649455e2e66d569ef9 (DEFAULT_TABLE table=test-table [id=48672352e2db4f7e98505bbbf9454b15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:11:19.868671 19269 tablet_service.cc:1467] Processing CreateTablet for tablet 2e8a690421f44a649455e2e66d569ef9 (DEFAULT_TABLE table=test-table [id=48672352e2db4f7e98505bbbf9454b15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:11:19.870198 19269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e8a690421f44a649455e2e66d569ef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:11:19.870220 18821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e8a690421f44a649455e2e66d569ef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:11:19.869923 19038 tablet_service.cc:1467] Processing CreateTablet for tablet 2e8a690421f44a649455e2e66d569ef9 (DEFAULT_TABLE table=test-table [id=48672352e2db4f7e98505bbbf9454b15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:11:19.871086 19038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e8a690421f44a649455e2e66d569ef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:11:19.883904 19399 tablet_bootstrap.cc:492] T 2e8a690421f44a649455e2e66d569ef9 P 2ee1436310174f20aa6e60f0e644cc93: Bootstrap starting.
I20260809 15:11:19.886993 19399 tablet_bootstrap.cc:654] T 2e8a690421f44a649455e2e66d569ef9 P 2ee1436310174f20aa6e60f0e644cc93: Neither blocks nor log segments found. Creating new log.
I20260809 15:11:19.888056 19399 log.cc:824] T 2e8a690421f44a649455e2e66d569ef9 P 2ee1436310174f20aa6e60f0e644cc93: Log is configured to *not* fsync() on all Append() calls
I20260809 15:11:19.891645 19401 tablet_bootstrap.cc:492] T 2e8a690421f44a649455e2e66d569ef9 P e6fbb915997b4f3da1537556271642f6: Bootstrap starting.
I20260809 15:11:19.892951 19403 tablet_bootstrap.cc:492] T 2e8a690421f44a649455e2e66d569ef9 P d40021626873405fb8183692867ac93f: Bootstrap starting.
I20260809 15:11:19.895454 19401 tablet_bootstrap.cc:654] T 2e8a690421f44a649455e2e66d569ef9 P e6fbb915997b4f3da1537556271642f6: Neither blocks nor log segments found. Creating new log.
I20260809 15:11:19.896456 19399 tablet_bootstrap.cc:492] T 2e8a690421f44a649455e2e66d569ef9 P 2ee1436310174f20aa6e60f0e644cc93: No bootstrap required, opened a new log
I20260809 15:11:19.896492 19401 log.cc:824] T 2e8a690421f44a649455e2e66d569ef9 P e6fbb915997b4f3da1537556271642f6: Log is configured to *not* fsync() on all Append() calls
I20260809 15:11:19.896731 19399 ts_tablet_manager.cc:1397] T 2e8a690421f44a649455e2e66d569ef9 P 2ee1436310174f20aa6e60f0e644cc93: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.007s
I20260809 15:11:19.897270 19403 tablet_bootstrap.cc:654] T 2e8a690421f44a649455e2e66d569ef9 P d40021626873405fb8183692867ac93f: Neither blocks nor log segments found. Creating new log.
I20260809 15:11:19.898466 19403 log.cc:824] T 2e8a690421f44a649455e2e66d569ef9 P d40021626873405fb8183692867ac93f: Log is configured to *not* fsync() on all Append() calls
I20260809 15:11:19.899369 19401 tablet_bootstrap.cc:492] T 2e8a690421f44a649455e2e66d569ef9 P e6fbb915997b4f3da1537556271642f6: No bootstrap required, opened a new log
I20260809 15:11:19.899642 19401 ts_tablet_manager.cc:1397] T 2e8a690421f44a649455e2e66d569ef9 P e6fbb915997b4f3da1537556271642f6: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260809 15:11:19.901542 19403 tablet_bootstrap.cc:492] T 2e8a690421f44a649455e2e66d569ef9 P d40021626873405fb8183692867ac93f: No bootstrap required, opened a new log
I20260809 15:11:19.901839 19403 ts_tablet_manager.cc:1397] T 2e8a690421f44a649455e2e66d569ef9 P d40021626873405fb8183692867ac93f: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260809 15:11:19.906780 19401 raft_consensus.cc:348] T 2e8a690421f44a649455e2e66d569ef9 P e6fbb915997b4f3da1537556271642f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ee1436310174f20aa6e60f0e644cc93" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 42299 } } peers { permanent_uuid: "d40021626873405fb8183692867ac93f" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38479 } } peers { permanent_uuid: "e6fbb915997b4f3da1537556271642f6" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 46463 } }
I20260809 15:11:19.906822 19399 raft_consensus.cc:348] T 2e8a690421f44a649455e2e66d569ef9 P 2ee1436310174f20aa6e60f0e644cc93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ee1436310174f20aa6e60f0e644cc93" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 42299 } } peers { permanent_uuid: "d40021626873405fb8183692867ac93f" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38479 } } peers { permanent_uuid: "e6fbb915997b4f3da1537556271642f6" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 46463 } }
I20260809 15:11:19.907122 19401 raft_consensus.cc:374] T 2e8a690421f44a649455e2e66d569ef9 P e6fbb915997b4f3da1537556271642f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:11:19.907208 19401 raft_consensus.cc:729] T 2e8a690421f44a649455e2e66d569ef9 P e6fbb915997b4f3da1537556271642f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6fbb915997b4f3da1537556271642f6, State: Initialized, Role: FOLLOWER
I20260809 15:11:19.907219 19399 raft_consensus.cc:374] T 2e8a690421f44a649455e2e66d569ef9 P 2ee1436310174f20aa6e60f0e644cc93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:11:19.907308 19399 raft_consensus.cc:729] T 2e8a690421f44a649455e2e66d569ef9 P 2ee1436310174f20aa6e60f0e644cc93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ee1436310174f20aa6e60f0e644cc93, State: Initialized, Role: FOLLOWER
I20260809 15:11:19.907881 19401 consensus_queue.cc:260] T 2e8a690421f44a649455e2e66d569ef9 P e6fbb915997b4f3da1537556271642f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ee1436310174f20aa6e60f0e644cc93" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 42299 } } peers { permanent_uuid: "d40021626873405fb8183692867ac93f" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38479 } } peers { permanent_uuid: "e6fbb915997b4f3da1537556271642f6" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 46463 } }
I20260809 15:11:19.907922 19399 consensus_queue.cc:260] T 2e8a690421f44a649455e2e66d569ef9 P 2ee1436310174f20aa6e60f0e644cc93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ee1436310174f20aa6e60f0e644cc93" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 42299 } } peers { permanent_uuid: "d40021626873405fb8183692867ac93f" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38479 } } peers { permanent_uuid: "e6fbb915997b4f3da1537556271642f6" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 46463 } }
I20260809 15:11:19.909560 19401 ts_tablet_manager.cc:1428] T 2e8a690421f44a649455e2e66d569ef9 P e6fbb915997b4f3da1537556271642f6: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260809 15:11:19.910441 19149 heartbeater.cc:503] Master 127.24.183.254:35881 was elected leader, sending a full tablet report...
I20260809 15:11:19.911005 19399 ts_tablet_manager.cc:1428] T 2e8a690421f44a649455e2e66d569ef9 P 2ee1436310174f20aa6e60f0e644cc93: Time spent starting tablet: real 0.014s user 0.014s sys 0.001s
I20260809 15:11:19.912005 19403 raft_consensus.cc:348] T 2e8a690421f44a649455e2e66d569ef9 P d40021626873405fb8183692867ac93f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ee1436310174f20aa6e60f0e644cc93" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 42299 } } peers { permanent_uuid: "d40021626873405fb8183692867ac93f" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38479 } } peers { permanent_uuid: "e6fbb915997b4f3da1537556271642f6" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 46463 } }
I20260809 15:11:19.912436 19403 raft_consensus.cc:374] T 2e8a690421f44a649455e2e66d569ef9 P d40021626873405fb8183692867ac93f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:11:19.912549 19403 raft_consensus.cc:729] T 2e8a690421f44a649455e2e66d569ef9 P d40021626873405fb8183692867ac93f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d40021626873405fb8183692867ac93f, State: Initialized, Role: FOLLOWER
I20260809 15:11:19.913148 19403 consensus_queue.cc:260] T 2e8a690421f44a649455e2e66d569ef9 P d40021626873405fb8183692867ac93f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ee1436310174f20aa6e60f0e644cc93" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 42299 } } peers { permanent_uuid: "d40021626873405fb8183692867ac93f" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38479 } } peers { permanent_uuid: "e6fbb915997b4f3da1537556271642f6" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 46463 } }
I20260809 15:11:19.914184 19369 heartbeater.cc:503] Master 127.24.183.254:35881 was elected leader, sending a full tablet report...
I20260809 15:11:19.916195 19403 ts_tablet_manager.cc:1428] T 2e8a690421f44a649455e2e66d569ef9 P d40021626873405fb8183692867ac93f: Time spent starting tablet: real 0.014s user 0.012s sys 0.001s
I20260809 15:11:19.949641 19410 raft_consensus.cc:482] T 2e8a690421f44a649455e2e66d569ef9 P d40021626873405fb8183692867ac93f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:11:19.949971 19410 raft_consensus.cc:504] T 2e8a690421f44a649455e2e66d569ef9 P d40021626873405fb8183692867ac93f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ee1436310174f20aa6e60f0e644cc93" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 42299 } } peers { permanent_uuid: "d40021626873405fb8183692867ac93f" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38479 } } peers { permanent_uuid: "e6fbb915997b4f3da1537556271642f6" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 46463 } }
I20260809 15:11:19.951629 19410 leader_election.cc:288] T 2e8a690421f44a649455e2e66d569ef9 P d40021626873405fb8183692867ac93f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ee1436310174f20aa6e60f0e644cc93 (127.24.183.194:42299), e6fbb915997b4f3da1537556271642f6 (127.24.183.193:46463)
I20260809 15:11:19.957062 19389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:19.952136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:34939 (local address 127.24.183.194:42299)
0809 15:11:19.952383 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0809 15:11:19.952389 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:11:19.952421 (+ 32us) server_negotiation.cc:407] Connection header received
0809 15:11:19.952495 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:11:19.952500 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:11:19.952576 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:11:19.952664 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:11:19.953095 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:19.953833 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:19.955409 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:19.956213 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:19.956365 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:11:19.956439 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 15:11:19.956686 (+ 247us) server_negotiation.cc:299] Negotiation successful
0809 15:11:19.956826 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 15:11:19.957063 19371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:19.951881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:42299 (local address 127.24.183.195:34939)
0809 15:11:19.952131 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:19.952153 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 15:11:19.952374 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:11:19.952707 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:11:19.952712 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:11:19.952741 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:11:19.952955 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:11:19.952965 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:11:19.953975 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:11:19.953978 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:11:19.955245 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:11:19.955260 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:11:19.956359 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:11:19.956365 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:11:19.956534 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:11:19.956559 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 15:11:19.956640 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 15:11:19.956772 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"mutex_wait_us":1}
I20260809 15:11:19.957958 19398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:19.952301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:47435 (local address 127.24.183.193:46463)
0809 15:11:19.952505 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 15:11:19.952511 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:11:19.953113 (+ 602us) server_negotiation.cc:407] Connection header received
0809 15:11:19.953275 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:11:19.953279 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:11:19.953339 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:11:19.953409 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:11:19.953907 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:19.954635 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:19.956308 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:19.957069 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:19.957218 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:11:19.957291 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 15:11:19.957649 (+ 358us) server_negotiation.cc:299] Negotiation successful
0809 15:11:19.957782 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 15:11:19.958045 19411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:19.952121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:46463 (local address 127.24.183.195:47435)
0809 15:11:19.953015 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:19.953029 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:11:19.953151 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:11:19.953434 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:11:19.953438 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:11:19.953461 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:11:19.953771 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:11:19.953779 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:11:19.954766 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:11:19.954771 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:11:19.956157 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:11:19.956168 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:11:19.957388 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:11:19.957393 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:11:19.957506 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:11:19.957528 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 15:11:19.957614 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 15:11:19.957740 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"thread_start_us":511,"threads_started":1}
I20260809 15:11:19.958122 19072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e8a690421f44a649455e2e66d569ef9" candidate_uuid: "d40021626873405fb8183692867ac93f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ee1436310174f20aa6e60f0e644cc93" is_pre_election: true
I20260809 15:11:19.958683 19072 raft_consensus.cc:2454] T 2e8a690421f44a649455e2e66d569ef9 P 2ee1436310174f20aa6e60f0e644cc93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d40021626873405fb8183692867ac93f in term 0.
I20260809 15:11:19.959055 18850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e8a690421f44a649455e2e66d569ef9" candidate_uuid: "d40021626873405fb8183692867ac93f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6fbb915997b4f3da1537556271642f6" is_pre_election: true
I20260809 15:11:19.959302 19183 leader_election.cc:302] T 2e8a690421f44a649455e2e66d569ef9 P d40021626873405fb8183692867ac93f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ee1436310174f20aa6e60f0e644cc93, d40021626873405fb8183692867ac93f; no voters:
I20260809 15:11:19.959506 18850 raft_consensus.cc:2454] T 2e8a690421f44a649455e2e66d569ef9 P e6fbb915997b4f3da1537556271642f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d40021626873405fb8183692867ac93f in term 0.
I20260809 15:11:19.959703 19410 raft_consensus.cc:2781] T 2e8a690421f44a649455e2e66d569ef9 P d40021626873405fb8183692867ac93f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:11:19.959857 19410 raft_consensus.cc:482] T 2e8a690421f44a649455e2e66d569ef9 P d40021626873405fb8183692867ac93f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:11:19.959969 19410 raft_consensus.cc:3037] T 2e8a690421f44a649455e2e66d569ef9 P d40021626873405fb8183692867ac93f [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:11:19.962257 19410 raft_consensus.cc:504] T 2e8a690421f44a649455e2e66d569ef9 P d40021626873405fb8183692867ac93f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ee1436310174f20aa6e60f0e644cc93" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 42299 } } peers { permanent_uuid: "d40021626873405fb8183692867ac93f" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38479 } } peers { permanent_uuid: "e6fbb915997b4f3da1537556271642f6" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 46463 } }
I20260809 15:11:19.962852 19410 leader_election.cc:288] T 2e8a690421f44a649455e2e66d569ef9 P d40021626873405fb8183692867ac93f [CANDIDATE]: Term 1 election: Requested vote from peers 2ee1436310174f20aa6e60f0e644cc93 (127.24.183.194:42299), e6fbb915997b4f3da1537556271642f6 (127.24.183.193:46463)
I20260809 15:11:19.963204 19072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e8a690421f44a649455e2e66d569ef9" candidate_uuid: "d40021626873405fb8183692867ac93f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ee1436310174f20aa6e60f0e644cc93"
I20260809 15:11:19.963266 18850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e8a690421f44a649455e2e66d569ef9" candidate_uuid: "d40021626873405fb8183692867ac93f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6fbb915997b4f3da1537556271642f6"
I20260809 15:11:19.963416 19072 raft_consensus.cc:3037] T 2e8a690421f44a649455e2e66d569ef9 P 2ee1436310174f20aa6e60f0e644cc93 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:11:19.963443 18850 raft_consensus.cc:3037] T 2e8a690421f44a649455e2e66d569ef9 P e6fbb915997b4f3da1537556271642f6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:11:19.966622 18850 raft_consensus.cc:2454] T 2e8a690421f44a649455e2e66d569ef9 P e6fbb915997b4f3da1537556271642f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d40021626873405fb8183692867ac93f in term 1.
I20260809 15:11:19.966626 19072 raft_consensus.cc:2454] T 2e8a690421f44a649455e2e66d569ef9 P 2ee1436310174f20aa6e60f0e644cc93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d40021626873405fb8183692867ac93f in term 1.
I20260809 15:11:19.967005 19183 leader_election.cc:302] T 2e8a690421f44a649455e2e66d569ef9 P d40021626873405fb8183692867ac93f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ee1436310174f20aa6e60f0e644cc93, d40021626873405fb8183692867ac93f; no voters:
I20260809 15:11:19.967293 19410 raft_consensus.cc:2781] T 2e8a690421f44a649455e2e66d569ef9 P d40021626873405fb8183692867ac93f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:11:19.967725 19410 raft_consensus.cc:686] T 2e8a690421f44a649455e2e66d569ef9 P d40021626873405fb8183692867ac93f [term 1 LEADER]: Becoming Leader. State: Replica: d40021626873405fb8183692867ac93f, State: Running, Role: LEADER
I20260809 15:11:19.968230 19410 consensus_queue.cc:237] T 2e8a690421f44a649455e2e66d569ef9 P d40021626873405fb8183692867ac93f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ee1436310174f20aa6e60f0e644cc93" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 42299 } } peers { permanent_uuid: "d40021626873405fb8183692867ac93f" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38479 } } peers { permanent_uuid: "e6fbb915997b4f3da1537556271642f6" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 46463 } }
I20260809 15:11:19.972613 18620 catalog_manager.cc:5495] T 2e8a690421f44a649455e2e66d569ef9 P d40021626873405fb8183692867ac93f reported cstate change: term changed from 0 to 1, leader changed from <none> to d40021626873405fb8183692867ac93f (127.24.183.195). New cstate: current_term: 1 leader_uuid: "d40021626873405fb8183692867ac93f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ee1436310174f20aa6e60f0e644cc93" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 42299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d40021626873405fb8183692867ac93f" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38479 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e6fbb915997b4f3da1537556271642f6" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 46463 } health_report { overall_health: UNKNOWN } } }
I20260809 15:11:20.016132 19394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:20.011117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35598 (local address 127.24.183.195:38479)
0809 15:11:20.011312 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 15:11:20.011317 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:11:20.011342 (+ 25us) server_negotiation.cc:407] Connection header received
0809 15:11:20.011498 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:11:20.011502 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:11:20.011583 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:11:20.011739 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:11:20.012116 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:20.012816 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:20.014117 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:20.014511 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:20.014596 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:11:20.015042 (+ 446us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:11:20.015583 (+ 541us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:11:20.015742 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 15:11:20.015834 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 15:11:20.015902 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 15:11:20.024194 18850 raft_consensus.cc:1261] T 2e8a690421f44a649455e2e66d569ef9 P e6fbb915997b4f3da1537556271642f6 [term 1 FOLLOWER]: Refusing update from remote peer d40021626873405fb8183692867ac93f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:11:20.024194 19072 raft_consensus.cc:1261] T 2e8a690421f44a649455e2e66d569ef9 P 2ee1436310174f20aa6e60f0e644cc93 [term 1 FOLLOWER]: Refusing update from remote peer d40021626873405fb8183692867ac93f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260809 15:11:20.024544 19370 tablet.cc:2369] T 2e8a690421f44a649455e2e66d569ef9 P d40021626873405fb8183692867ac93f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:11:20.025386 19410 consensus_queue.cc:1035] T 2e8a690421f44a649455e2e66d569ef9 P d40021626873405fb8183692867ac93f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6fbb915997b4f3da1537556271642f6" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 46463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:11:20.025760 19412 consensus_queue.cc:1035] T 2e8a690421f44a649455e2e66d569ef9 P d40021626873405fb8183692867ac93f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ee1436310174f20aa6e60f0e644cc93" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 42299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:11:20.036336 19417 mvcc.cc:205] Tried to move back new op lower bound from 7316636794968891392 to 7316636794753736704. Current Snapshot: MvccSnapshot[applied={T|T < 7316636794968891392}]
I20260809 15:11:20.051610 18620 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:11:20.051923 18620 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:11:20.055116 18620 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50: Sending DeleteTablet for 3 replicas of tablet 2e8a690421f44a649455e2e66d569ef9
I20260809 15:11:20.056202 19038 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e8a690421f44a649455e2e66d569ef9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:11:20 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:37996
I20260809 15:11:20.056252 19269 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e8a690421f44a649455e2e66d569ef9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:11:20 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:35588
I20260809 15:11:20.056903 18821 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e8a690421f44a649455e2e66d569ef9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:11:20 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:57314
I20260809 15:11:20.057157 19432 tablet_replica.cc:330] T 2e8a690421f44a649455e2e66d569ef9 P 2ee1436310174f20aa6e60f0e644cc93: stopping tablet replica
I20260809 15:11:20.057591 19432 raft_consensus.cc:2229] T 2e8a690421f44a649455e2e66d569ef9 P 2ee1436310174f20aa6e60f0e644cc93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:11:20.058104 19432 raft_consensus.cc:2258] T 2e8a690421f44a649455e2e66d569ef9 P 2ee1436310174f20aa6e60f0e644cc93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:11:20.058257 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 18688
I20260809 15:11:20.060407 19432 ts_tablet_manager.cc:1905] T 2e8a690421f44a649455e2e66d569ef9 P 2ee1436310174f20aa6e60f0e644cc93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:11:20.067972 19432 ts_tablet_manager.cc:1918] T 2e8a690421f44a649455e2e66d569ef9 P 2ee1436310174f20aa6e60f0e644cc93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:11:20.068135 19432 log.cc:1196] T 2e8a690421f44a649455e2e66d569ef9 P 2ee1436310174f20aa6e60f0e644cc93: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/wal/wals/2e8a690421f44a649455e2e66d569ef9
I20260809 15:11:20.068679 19432 ts_tablet_manager.cc:1939] T 2e8a690421f44a649455e2e66d569ef9 P 2ee1436310174f20aa6e60f0e644cc93: Deleting consensus metadata
I20260809 15:11:20.070648 18598 catalog_manager.cc:4841] TS 2ee1436310174f20aa6e60f0e644cc93 (127.24.183.194:42299): tablet 2e8a690421f44a649455e2e66d569ef9 (table test-table [id=48672352e2db4f7e98505bbbf9454b15]) successfully deleted
I20260809 15:11:20.071568 19435 tablet_replica.cc:330] T 2e8a690421f44a649455e2e66d569ef9 P e6fbb915997b4f3da1537556271642f6: stopping tablet replica
I20260809 15:11:20.072074 19435 raft_consensus.cc:2229] T 2e8a690421f44a649455e2e66d569ef9 P e6fbb915997b4f3da1537556271642f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:11:20.072669 19435 raft_consensus.cc:2258] T 2e8a690421f44a649455e2e66d569ef9 P e6fbb915997b4f3da1537556271642f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:11:20.073444 19431 tablet_replica.cc:330] T 2e8a690421f44a649455e2e66d569ef9 P d40021626873405fb8183692867ac93f: stopping tablet replica
I20260809 15:11:20.073860 19431 raft_consensus.cc:2229] T 2e8a690421f44a649455e2e66d569ef9 P d40021626873405fb8183692867ac93f [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:11:20.074525 19431 raft_consensus.cc:2258] T 2e8a690421f44a649455e2e66d569ef9 P d40021626873405fb8183692867ac93f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:11:20.075249 19435 ts_tablet_manager.cc:1905] T 2e8a690421f44a649455e2e66d569ef9 P e6fbb915997b4f3da1537556271642f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:11:20.076645 19431 ts_tablet_manager.cc:1905] T 2e8a690421f44a649455e2e66d569ef9 P d40021626873405fb8183692867ac93f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:11:20.081370 19398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:20.059762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57326 (local address 127.24.183.193:46463)
0809 15:11:20.059930 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 15:11:20.059937 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:11:20.059955 (+ 18us) server_negotiation.cc:407] Connection header received
0809 15:11:20.060010 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:11:20.060015 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:11:20.060093 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:11:20.060189 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:11:20.062319 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:20.063469 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:20.065545 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:20.066235 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:20.066358 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:11:20.072107 (+ 5749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:11:20.072121 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:11:20.072138 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:11:20.072197 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:11:20.076838 (+ 4641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:11:20.077827 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:11:20.077836 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:11:20.077840 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:11:20.078018 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:11:20.078492 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:11:20.078497 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:11:20.078501 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:11:20.078965 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:11:20.079301 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0809 15:11:20.080670 (+ 1369us) server_negotiation.cc:299] Negotiation successful
0809 15:11:20.080960 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 15:11:20.082330 18916 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:11:20.083963 19431 ts_tablet_manager.cc:1918] T 2e8a690421f44a649455e2e66d569ef9 P d40021626873405fb8183692867ac93f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:11:20.084139 19431 log.cc:1196] T 2e8a690421f44a649455e2e66d569ef9 P d40021626873405fb8183692867ac93f: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/wal/wals/2e8a690421f44a649455e2e66d569ef9
I20260809 15:11:20.084569 19431 ts_tablet_manager.cc:1939] T 2e8a690421f44a649455e2e66d569ef9 P d40021626873405fb8183692867ac93f: Deleting consensus metadata
I20260809 15:11:20.086053 18599 catalog_manager.cc:4841] TS d40021626873405fb8183692867ac93f (127.24.183.195:38479): tablet 2e8a690421f44a649455e2e66d569ef9 (table test-table [id=48672352e2db4f7e98505bbbf9454b15]) successfully deleted
I20260809 15:11:20.173600 19435 ts_tablet_manager.cc:1918] T 2e8a690421f44a649455e2e66d569ef9 P e6fbb915997b4f3da1537556271642f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:11:20.173885 19435 log.cc:1196] T 2e8a690421f44a649455e2e66d569ef9 P e6fbb915997b4f3da1537556271642f6: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/wal/wals/2e8a690421f44a649455e2e66d569ef9
I20260809 15:11:20.174448 19435 ts_tablet_manager.cc:1939] T 2e8a690421f44a649455e2e66d569ef9 P e6fbb915997b4f3da1537556271642f6: Deleting consensus metadata
I20260809 15:11:20.174758 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 18688
I20260809 15:11:20.204821 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 18934
W20260809 15:11:20.204977 18599 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.183.193:46463 (error 108)
I20260809 15:11:20.207054 19393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:20.206488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:46463 (local address 127.0.0.1:57332)
0809 15:11:20.206754 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:20.206875 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
W20260809 15:11:20.207332 18599 catalog_manager.cc:4584] TS e6fbb915997b4f3da1537556271642f6 (127.24.183.193:46463): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2e8a690421f44a649455e2e66d569ef9: Network error: Client connection negotiation failed: client connection to 127.24.183.193:46463: connect: Connection refused (error 111)
I20260809 15:11:20.224728 19389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:20.206017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38010 (local address 127.24.183.194:42299)
0809 15:11:20.206193 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 15:11:20.206199 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:11:20.206232 (+ 33us) server_negotiation.cc:407] Connection header received
0809 15:11:20.206306 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:11:20.206312 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:11:20.206406 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:11:20.206516 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:11:20.208260 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:20.209414 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:20.210708 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:20.211352 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:20.211479 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:11:20.217927 (+ 6448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:11:20.217948 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:11:20.217968 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:11:20.218011 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:11:20.221653 (+ 3642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:11:20.222370 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:11:20.222380 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:11:20.222384 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:11:20.222533 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:11:20.222872 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:11:20.222877 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:11:20.222880 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:11:20.223374 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:11:20.223747 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0809 15:11:20.224110 (+ 363us) server_negotiation.cc:299] Negotiation successful
0809 15:11:20.224354 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 15:11:20.225492 19137 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:11:20.261066 19393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:20.260477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:46463 (local address 127.0.0.1:57346)
0809 15:11:20.260735 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:20.260855 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260809 15:11:20.262337 19393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:20.261914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:46463 (local address 127.0.0.1:57350)
0809 15:11:20.262076 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:20.262185 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 15:11:20.315896 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 18934
I20260809 15:11:20.323078 19393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:20.322523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:46463 (local address 127.0.0.1:57364)
0809 15:11:20.322759 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:20.322881 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 15:11:20.324333 19393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:20.323930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:46463 (local address 127.0.0.1:57370)
0809 15:11:20.324108 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:20.324192 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 15:11:20.340452 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 19157
I20260809 15:11:20.354859 19394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:20.341189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35612 (local address 127.24.183.195:38479)
0809 15:11:20.341374 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 15:11:20.341379 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:11:20.341601 (+ 222us) server_negotiation.cc:407] Connection header received
0809 15:11:20.341775 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:11:20.341780 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:11:20.341866 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:11:20.341950 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:11:20.343292 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:20.344174 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:20.345593 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:20.346043 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:20.346158 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:11:20.348834 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:11:20.348847 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:11:20.348865 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:11:20.348908 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:11:20.351974 (+ 3066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:11:20.352632 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:11:20.352640 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:11:20.352642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:11:20.352808 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:11:20.353181 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:11:20.353185 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:11:20.353187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:11:20.353537 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:11:20.353844 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0809 15:11:20.354142 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 15:11:20.354362 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 15:11:20.355523 19362 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:11:20.404966 19393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:20.404404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:46463 (local address 127.0.0.1:57374)
0809 15:11:20.404647 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:20.404775 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260809 15:11:20.406147 19393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:20.405687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:46463 (local address 127.0.0.1:57386)
0809 15:11:20.405921 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:20.406010 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
I20260809 15:11:20.479713 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 19157
I20260809 15:11:20.503960 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 18577
I20260809 15:11:20.505175 18675 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:11:20.622052 19393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:20.621385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:46463 (local address 127.0.0.1:57390)
0809 15:11:20.621633 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:20.621746 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:46463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260809 15:11:20.622026 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 18577
I20260809 15:11:40.652434 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:35881
--webserver_interface=127.24.183.254
--webserver_port=36299
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:35177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:35881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:11:40.989619 19450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:11:40.989930 19450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:11:40.990065 19450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:11:40.997208 19450 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 15:11:40.997308 19450 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 15:11:40.997362 19450 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 15:11:40.997433 19450 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:11:40.997509 19450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:11:40.997610 19450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:11:40.997669 19450 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:11:40.997745 19450 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:11:41.007259 19450 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:35177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:35881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:35881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=36299
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:11:41.008513 19450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:11:41.009838 19450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:11:41.021579 19456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:11:41.021618 19461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:11:41.021539 19457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:11:41.022356 19450 server_base.cc:1034] running on GCE node
I20260809 15:11:41.023063 19450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:11:41.024456 19450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:11:41.025673 19450 hybrid_clock.cc:648] HybridClock initialized: now 1786288301025604 us; error 43 us; skew 500 ppm
I20260809 15:11:41.025978 19450 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:11:41 dist-test-slave-nj21 krb5kdc[17634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288301, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:11:41.032076 19450 init.cc:377] Logged in from keytab as kudu/127.24.183.254@KRBTEST.COM (short username kudu)
I20260809 15:11:41.033456 19450 webserver.cc:460] Webserver started at http://127.24.183.254:36299/ using document root <none> and password file <none>
I20260809 15:11:41.034128 19450 fs_manager.cc:359] Metadata directory not provided
I20260809 15:11:41.034230 19450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:11:41.039837 19450 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 15:11:41.042527 19469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:11:41.043829 19450 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:11:41.043993 19450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "e1f1911f3a9b41b88d359babf89cda50"
format_stamp: "Formatted at 2026-08-09 15:10:55 on dist-test-slave-nj21"
server_key: "84af479603da0ec7563a902ff48c2210"
server_key_iv: "3412991e4a96584bea80206364fe9bd3"
server_key_version: "encryptionkey@0"
I20260809 15:11:41.044929 19450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:11:41.063287 19450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:11:41.067330 19450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:11:41.067579 19450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:11:41.085110 19450 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:35881
I20260809 15:11:41.085129 19549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:35881 every 8 connection(s)
I20260809 15:11:41.086540 19450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:11:41.089910 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 19450
I20260809 15:11:41.091109 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.193:46463
--local_ip_for_outbound_sockets=127.24.183.193
--tserver_master_addrs=127.24.183.254:35881
--webserver_port=41175
--webserver_interface=127.24.183.193
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:35177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:11:41.095258 19550 sys_catalog.cc:263] Verifying existing consensus state
I20260809 15:11:41.099838 19550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50: Bootstrap starting.
I20260809 15:11:41.122725 19550 log.cc:824] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50: Log is configured to *not* fsync() on all Append() calls
I20260809 15:11:41.140954 19550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:11:41.141551 19550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50: Bootstrap complete.
I20260809 15:11:41.151482 19550 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f1911f3a9b41b88d359babf89cda50" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 35881 } }
I20260809 15:11:41.152462 19550 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f1911f3a9b41b88d359babf89cda50, State: Initialized, Role: FOLLOWER
I20260809 15:11:41.153122 19550 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f1911f3a9b41b88d359babf89cda50" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 35881 } }
I20260809 15:11:41.153391 19550 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:11:41.153522 19550 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:11:41.153715 19550 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:11:41.157719 19550 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f1911f3a9b41b88d359babf89cda50" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 35881 } }
I20260809 15:11:41.158223 19550 leader_election.cc:302] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1f1911f3a9b41b88d359babf89cda50; no voters:
I20260809 15:11:41.158905 19550 leader_election.cc:288] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 15:11:41.159065 19555 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:11:41.160128 19555 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [term 3 LEADER]: Becoming Leader. State: Replica: e1f1911f3a9b41b88d359babf89cda50, State: Running, Role: LEADER
I20260809 15:11:41.160727 19555 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f1911f3a9b41b88d359babf89cda50" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 35881 } }
I20260809 15:11:41.161159 19550 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:11:41.163120 19560 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1f1911f3a9b41b88d359babf89cda50. Latest consensus state: current_term: 3 leader_uuid: "e1f1911f3a9b41b88d359babf89cda50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f1911f3a9b41b88d359babf89cda50" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 35881 } } }
I20260809 15:11:41.163482 19560 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [sys.catalog]: This master's current role is: LEADER
I20260809 15:11:41.169116 19557 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "e1f1911f3a9b41b88d359babf89cda50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f1911f3a9b41b88d359babf89cda50" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 35881 } } }
I20260809 15:11:41.169456 19557 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50 [sys.catalog]: This master's current role is: LEADER
I20260809 15:11:41.170341 19564 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:11:41.179512 19564 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:11:41.181120 19564 catalog_manager.cc:1259] Loaded cluster ID: 9ff381450f124676b3c29c4fed0a7ac4
I20260809 15:11:41.181301 19564 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:11:41.185784 19564 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:11:41.187589 19564 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50: Loaded TSK: 0
I20260809 15:11:41.187849 19564 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e1f1911f3a9b41b88d359babf89cda50: Loaded TSK: 1
F20260809 15:11:41.189085 19564 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f15177ff9ad google::LogMessage::Fail() at ??:0
@ 0x7f151780288a google::LogMessage::SendToLog() at ??:0
@ 0x7f15177ff4de google::LogMessage::Flush() at ??:0
@ 0x7f1517800a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f151b5637d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f151b55273e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f152d22e1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f152d22d783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f152d2b92a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f152d2b90c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f152d3035e9 std::function<>::operator()() at ??:0
@ 0x7f152d231cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f152d22fc36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f152d2b7610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f152d2b7439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f151a44b345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f151a44e91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f151a44e799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f151a429eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f151d7036db start_thread at ??:0
@ 0x7f151669961f clone at ??:0
W20260809 15:11:41.479163 19552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:11:41.479417 19552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:11:41.479548 19552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:11:41.486675 19552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:11:41.486786 19552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:11:41.486945 19552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.193
I20260809 15:11:41.496519 19552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:35177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.193:46463
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.183.193
--webserver_port=41175
--tserver_master_addrs=127.24.183.254:35881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.193
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:11:41.497728 19552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:11:41.499018 19552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:11:41.511653 19630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:11:41.511616 19631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:11:41.511698 19634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:11:41.512763 19552 server_base.cc:1034] running on GCE node
I20260809 15:11:41.513307 19552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:11:41.514163 19552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:11:41.515350 19552 hybrid_clock.cc:648] HybridClock initialized: now 1786288301515304 us; error 37 us; skew 500 ppm
I20260809 15:11:41.515705 19552 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:11:41 dist-test-slave-nj21 krb5kdc[17634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288301, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:11:41.518855 19552 init.cc:377] Logged in from keytab as kudu/127.24.183.193@KRBTEST.COM (short username kudu)
I20260809 15:11:41.520299 19552 webserver.cc:460] Webserver started at http://127.24.183.193:41175/ using document root <none> and password file <none>
I20260809 15:11:41.520965 19552 fs_manager.cc:359] Metadata directory not provided
I20260809 15:11:41.521077 19552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:11:41.526410 19552 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 15:11:41.529137 19644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:11:41.530465 19552 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:11:41.530622 19552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/wal
uuid: "e6fbb915997b4f3da1537556271642f6"
format_stamp: "Formatted at 2026-08-09 15:10:55 on dist-test-slave-nj21"
server_key: "12c0b4701e3adf4c3c503c274d80b43a"
server_key_iv: "44e535ee8626ae52197aabd6f2d29f1d"
server_key_version: "encryptionkey@0"
I20260809 15:11:41.531584 19552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:11:41.550467 19552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:11:41.554620 19552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:11:41.554872 19552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:11:41.555863 19552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:11:41.557189 19552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:11:41.557286 19552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:11:41.557401 19552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:11:41.557475 19552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:11:41.589111 19552 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.193:46463
I20260809 15:11:41.589179 19837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.193:46463 every 8 connection(s)
I20260809 15:11:41.590569 19552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
I20260809 15:11:41.592015 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 19552
I20260809 15:11:41.592795 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.194:42299
--local_ip_for_outbound_sockets=127.24.183.194
--tserver_master_addrs=127.24.183.254:35881
--webserver_port=36033
--webserver_interface=127.24.183.194
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:35177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:11:41.595959 19841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:41.593682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.193:53217)
0809 15:11:41.594552 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:41.594816 (+ 264us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":315,"threads_started":1}
W20260809 15:11:41.596632 19838 heartbeater.cc:644] Failed to heartbeat to 127.24.183.254:35881 (0 consecutive failures): Network error: Failed to ping master at 127.24.183.254:35881: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
I20260809 15:11:41.597472 19841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:41.597115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.193:47991)
0809 15:11:41.597262 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:41.597349 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 15:11:41.598284 19841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:41.597942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.193:41431)
0809 15:11:41.598069 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:41.598151 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 15:11:41.598505 19838 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 15:11:41.930729 19842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:11:41.931015 19842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:11:41.931151 19842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:11:41.938696 19842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:11:41.938803 19842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:11:41.938954 19842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.194
I20260809 15:11:41.948493 19842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:35177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.194:42299
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.183.194
--webserver_port=36033
--tserver_master_addrs=127.24.183.254:35881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.194
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:11:41.949834 19842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:11:41.951151 19842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:11:41.963800 19853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:11:41.963845 19857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:11:41.963757 19855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:11:41.964583 19842 server_base.cc:1034] running on GCE node
I20260809 15:11:41.965162 19842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:11:41.966035 19842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:11:41.967231 19842 hybrid_clock.cc:648] HybridClock initialized: now 1786288301967183 us; error 38 us; skew 500 ppm
I20260809 15:11:41.967523 19842 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:11:41 dist-test-slave-nj21 krb5kdc[17634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288301, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:11:41.970451 19842 init.cc:377] Logged in from keytab as kudu/127.24.183.194@KRBTEST.COM (short username kudu)
I20260809 15:11:41.971863 19842 webserver.cc:460] Webserver started at http://127.24.183.194:36033/ using document root <none> and password file <none>
I20260809 15:11:41.972499 19842 fs_manager.cc:359] Metadata directory not provided
I20260809 15:11:41.972620 19842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:11:41.977636 19842 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.004s
I20260809 15:11:41.980207 19870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:11:41.981436 19842 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:11:41.981588 19842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/wal
uuid: "2ee1436310174f20aa6e60f0e644cc93"
format_stamp: "Formatted at 2026-08-09 15:10:56 on dist-test-slave-nj21"
server_key: "5b45486b0132c29cdb81515d61c07361"
server_key_iv: "86e6e5440eda4ded60d4be036963a159"
server_key_version: "encryptionkey@0"
I20260809 15:11:41.982528 19842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:11:42.002130 19842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:11:42.006067 19842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:11:42.006330 19842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:11:42.007331 19842 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:11:42.008718 19842 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:11:42.008826 19842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:11:42.008939 19842 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:11:42.009009 19842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:11:42.040172 19842 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.194:42299
I20260809 15:11:42.040242 20048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.194:42299 every 8 connection(s)
I20260809 15:11:42.041639 19842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
I20260809 15:11:42.046432 20052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:42.044626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.194:54027)
0809 15:11:42.045364 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:42.045579 (+ 215us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":285,"threads_started":1}
W20260809 15:11:42.047032 20050 heartbeater.cc:644] Failed to heartbeat to 127.24.183.254:35881 (0 consecutive failures): Network error: Failed to ping master at 127.24.183.254:35881: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
I20260809 15:11:42.047797 20052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:42.047465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.194:41831)
0809 15:11:42.047581 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:42.047643 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 15:11:42.048511 20052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:42.048232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.194:37037)
0809 15:11:42.048335 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:42.048411 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 15:11:42.048717 20050 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 15:11:42.051038 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 19842
I20260809 15:11:42.051638 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.195:38479
--local_ip_for_outbound_sockets=127.24.183.195
--tserver_master_addrs=127.24.183.254:35881
--webserver_port=44075
--webserver_interface=127.24.183.195
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:35177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:11:42.383245 20054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:11:42.383507 20054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:11:42.383636 20054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:11:42.390925 20054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:11:42.391027 20054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:11:42.391175 20054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.195
I20260809 15:11:42.400719 20054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:35177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.195:38479
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.183.195
--webserver_port=44075
--tserver_master_addrs=127.24.183.254:35881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.195
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:11:42.401999 20054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:11:42.403275 20054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:11:42.415828 20060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:11:42.415858 20064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:11:42.415848 20066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:11:42.416712 20054 server_base.cc:1034] running on GCE node
I20260809 15:11:42.417294 20054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:11:42.418167 20054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:11:42.419359 20054 hybrid_clock.cc:648] HybridClock initialized: now 1786288302419315 us; error 34 us; skew 500 ppm
I20260809 15:11:42.419687 20054 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:11:42 dist-test-slave-nj21 krb5kdc[17634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288302, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:11:42.422685 20054 init.cc:377] Logged in from keytab as kudu/127.24.183.195@KRBTEST.COM (short username kudu)
I20260809 15:11:42.424081 20054 webserver.cc:460] Webserver started at http://127.24.183.195:44075/ using document root <none> and password file <none>
I20260809 15:11:42.424749 20054 fs_manager.cc:359] Metadata directory not provided
I20260809 15:11:42.424871 20054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:11:42.430049 20054 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 15:11:42.432690 20082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:11:42.433943 20054 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:11:42.434095 20054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/wal
uuid: "d40021626873405fb8183692867ac93f"
format_stamp: "Formatted at 2026-08-09 15:10:56 on dist-test-slave-nj21"
server_key: "ea035a8094f632db2b52fe36227f7a2e"
server_key_iv: "fc5815666a05ebee49041246e29646ae"
server_key_version: "encryptionkey@0"
I20260809 15:11:42.435091 20054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:11:42.455570 20054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:11:42.459494 20054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:11:42.459765 20054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:11:42.460741 20054 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:11:42.462071 20054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:11:42.462170 20054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:11:42.462286 20054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:11:42.462357 20054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:11:42.493062 20054 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.195:38479
I20260809 15:11:42.493120 20256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.195:38479 every 8 connection(s)
I20260809 15:11:42.494469 20054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
I20260809 15:11:42.499011 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 20054
I20260809 15:11:42.499210 20262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:42.497183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.195:59715)
0809 15:11:42.497923 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:42.498149 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":269,"threads_started":1}
W20260809 15:11:42.499924 20258 heartbeater.cc:644] Failed to heartbeat to 127.24.183.254:35881 (0 consecutive failures): Network error: Failed to ping master at 127.24.183.254:35881: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
I20260809 15:11:42.500774 20262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:42.500432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.195:43463)
0809 15:11:42.500571 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:42.500645 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 15:11:42.501564 20262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:42.501243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.195:60463)
0809 15:11:42.501373 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:42.501456 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 15:11:42.501775 20258 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 15:11:42.599761 20264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:42.598953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.193:51117)
0809 15:11:42.599400 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:42.599498 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":216,"threads_started":1}
I20260809 15:11:43.050004 20265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:43.049218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.194:39505)
0809 15:11:43.049680 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:43.049778 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":239,"threads_started":1}
I20260809 15:11:43.503090 20268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:43.502302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.195:52215)
0809 15:11:43.502774 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:43.502864 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":225,"threads_started":1}
I20260809 15:11:43.601289 20269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:43.600554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.193:39739)
0809 15:11:43.601032 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:43.601115 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":233,"threads_started":1}
I20260809 15:11:44.051476 20270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:44.050809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.194:52071)
0809 15:11:44.051231 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:44.051311 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":207,"threads_started":1}
I20260809 15:11:44.504648 20271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:44.503934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.195:46861)
0809 15:11:44.504400 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:44.504482 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":227,"threads_started":1}
I20260809 15:11:44.602972 20272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:44.602227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.193:51213)
0809 15:11:44.602696 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:44.602785 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":250,"threads_started":1}
I20260809 15:11:45.052964 20273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:45.052278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.194:51539)
0809 15:11:45.052717 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:45.052798 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":240,"threads_started":1}
I20260809 15:11:45.506315 20274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:45.505557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.195:48321)
0809 15:11:45.506045 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:45.506132 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":246,"threads_started":1}
I20260809 15:11:45.604616 20275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:45.603883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.193:45785)
0809 15:11:45.604337 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:45.604423 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":223,"threads_started":1}
I20260809 15:11:46.054738 20276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:46.053910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.194:48889)
0809 15:11:46.054441 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:46.054535 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":253,"threads_started":1}
I20260809 15:11:46.508018 20277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:46.507150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.195:50273)
0809 15:11:46.507659 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:46.507790 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":246,"threads_started":1}
I20260809 15:11:46.606109 20278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:46.605438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.193:43347)
0809 15:11:46.605861 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:46.605942 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":205,"threads_started":1}
I20260809 15:11:47.056408 20279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:47.055585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.194:33721)
0809 15:11:47.056119 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:47.056224 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":275,"threads_started":1}
I20260809 15:11:47.509569 20282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:47.508831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.195:41505)
0809 15:11:47.509297 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:47.509376 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":235,"threads_started":1}
I20260809 15:11:47.607550 20283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:47.606888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.193:40893)
0809 15:11:47.607298 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:47.607387 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":205,"threads_started":1}
I20260809 15:11:48.057951 20284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:48.057241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.194:50545)
0809 15:11:48.057685 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:48.057777 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":229,"threads_started":1}
I20260809 15:11:48.511047 20285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:48.510339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.195:58525)
0809 15:11:48.510782 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:48.510869 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":226,"threads_started":1}
I20260809 15:11:48.609233 20286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:48.608441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.193:36173)
0809 15:11:48.608927 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:48.609010 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":217,"threads_started":1}
I20260809 15:11:49.059753 20288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:49.058882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.194:57819)
0809 15:11:49.059392 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:49.059476 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":209,"threads_started":1}
I20260809 15:11:49.512843 20292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:49.511922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.195:51203)
0809 15:11:49.512460 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:49.512554 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":226,"threads_started":1}
I20260809 15:11:49.610802 20293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:49.610081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.193:39349)
0809 15:11:49.610531 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:49.610621 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":224,"threads_started":1}
I20260809 15:11:50.061311 20295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:50.060567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.194:58677)
0809 15:11:50.061046 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:50.061136 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":244,"threads_started":1}
I20260809 15:11:50.514443 20296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:50.513708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.195:47753)
0809 15:11:50.514163 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:50.514264 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":230,"threads_started":1}
I20260809 15:11:50.612407 20297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:50.611630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.193:33835)
0809 15:11:50.612126 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:50.612210 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":236,"threads_started":1}
I20260809 15:11:51.063114 20298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:51.062139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.194:40167)
0809 15:11:51.062737 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:51.062859 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":294,"threads_started":1}
I20260809 15:11:51.516171 20300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:51.515294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.195:34913)
0809 15:11:51.515893 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:51.515985 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":279,"threads_started":1}
I20260809 15:11:51.614271 20301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:51.613455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.193:53709)
0809 15:11:51.614000 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:51.614092 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":302,"threads_started":1}
I20260809 15:11:52.064977 20303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:52.064178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.194:52245)
0809 15:11:52.064678 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:52.064768 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":281,"threads_started":1}
I20260809 15:11:52.517879 20305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:52.517062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.195:57697)
0809 15:11:52.517544 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:52.517653 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":237,"threads_started":1}
I20260809 15:11:52.615865 20306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:52.615110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.193:50621)
0809 15:11:52.615557 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:52.615651 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":217,"threads_started":1}
I20260809 15:11:53.066658 20308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:53.065875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.194:50403)
0809 15:11:53.066365 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:53.066462 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":244,"threads_started":1}
I20260809 15:11:53.519524 20309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:53.518803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.195:52609)
0809 15:11:53.519250 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:53.519346 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":222,"threads_started":1}
I20260809 15:11:53.617434 20310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:53.616697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.193:41151)
0809 15:11:53.617164 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:53.617257 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":218,"threads_started":1}
I20260809 15:11:54.068296 20311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:54.067581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.194:47097)
0809 15:11:54.068047 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:54.068134 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":253,"threads_started":1}
I20260809 15:11:54.521083 20312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:54.520421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.195:51003)
0809 15:11:54.520843 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:54.520925 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":209,"threads_started":1}
I20260809 15:11:54.619138 20318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:54.618260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.193:43587)
0809 15:11:54.618809 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:54.618920 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":250,"threads_started":1}
I20260809 15:11:55.070029 20319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:55.069210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.194:55991)
0809 15:11:55.069731 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:55.069831 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":257,"threads_started":1}
I20260809 15:11:55.522696 20320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:55.521921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.195:60731)
0809 15:11:55.522409 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:55.522504 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":244,"threads_started":1}
I20260809 15:11:55.620759 20321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:55.620021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.193:59631)
0809 15:11:55.620456 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:55.620540 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":214,"threads_started":1}
I20260809 15:11:56.071827 20324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:56.070948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.194:34541)
0809 15:11:56.071447 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:56.071542 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":265,"threads_started":1}
I20260809 15:11:56.524290 20325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:56.523536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.195:38553)
0809 15:11:56.524014 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:56.524096 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":263,"threads_started":1}
I20260809 15:11:56.622576 20326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:56.621839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.193:44747)
0809 15:11:56.622275 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:56.622375 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":214,"threads_started":1}
I20260809 15:11:57.073748 20327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:57.072935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.194:36203)
0809 15:11:57.073427 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:57.073533 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":249,"threads_started":1}
I20260809 15:11:57.503469 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 19552
I20260809 15:11:57.521329 20328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:57.504687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37812 (local address 127.24.183.193:46463)
0809 15:11:57.505421 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0809 15:11:57.505441 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 15:11:57.505480 (+ 39us) server_negotiation.cc:407] Connection header received
0809 15:11:57.507173 (+ 1693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:11:57.507182 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:11:57.507604 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:11:57.507988 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:11:57.509483 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:57.510733 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:57.512009 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:57.512654 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:57.512851 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:11:57.515464 (+ 2613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:11:57.515472 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:11:57.515486 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:11:57.515524 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:11:57.518213 (+ 2689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:11:57.518832 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:11:57.518838 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:11:57.518844 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:11:57.519020 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:11:57.519379 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:11:57.519383 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:11:57.519385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:11:57.519928 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:11:57.520211 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0809 15:11:57.520644 (+ 433us) server_negotiation.cc:299] Negotiation successful
0809 15:11:57.521034 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":405,"threads_started":1}
I20260809 15:11:57.522632 19829 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:11:57.526170 20332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:57.525342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:35881 (local address 127.24.183.195:55565)
0809 15:11:57.525816 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0809 15:11:57.525952 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.254:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":244,"threads_started":1}
I20260809 15:11:57.605762 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 19552
I20260809 15:11:57.627019 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 19842
I20260809 15:11:57.644686 20333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:57.627961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44702 (local address 127.24.183.194:42299)
0809 15:11:57.628536 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0809 15:11:57.628552 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 15:11:57.628591 (+ 39us) server_negotiation.cc:407] Connection header received
0809 15:11:57.630327 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:11:57.630337 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:11:57.630770 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:11:57.631099 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:11:57.632755 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:57.633988 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:57.635297 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:57.636010 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:57.636180 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:11:57.638797 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:11:57.638804 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:11:57.638818 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:11:57.638878 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:11:57.641721 (+ 2843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:11:57.642352 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:11:57.642357 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:11:57.642363 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:11:57.642525 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:11:57.642809 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:11:57.642812 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:11:57.642814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:11:57.643304 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:11:57.643523 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 15:11:57.644086 (+ 563us) server_negotiation.cc:299] Negotiation successful
0809 15:11:57.644406 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":269,"threads_started":1}
I20260809 15:11:57.646075 20043 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:11:57.737609 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 19842
I20260809 15:11:57.762169 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 20054
I20260809 15:11:57.779623 20335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:57.763268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43516 (local address 127.24.183.195:38479)
0809 15:11:57.763975 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0809 15:11:57.763990 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 15:11:57.764036 (+ 46us) server_negotiation.cc:407] Connection header received
0809 15:11:57.765712 (+ 1676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:11:57.765724 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:11:57.766144 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:11:57.766444 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:11:57.767879 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:57.769121 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:57.770584 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:57.771240 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:57.771385 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:11:57.774034 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:11:57.774042 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:11:57.774057 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:11:57.774095 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:11:57.776793 (+ 2698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:11:57.777400 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:11:57.777406 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:11:57.777412 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:11:57.777579 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:11:57.777858 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:11:57.777860 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:11:57.777862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:11:57.778380 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:11:57.778614 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0809 15:11:57.778920 (+ 306us) server_negotiation.cc:299] Negotiation successful
0809 15:11:57.779300 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":345,"threads_started":1}
I20260809 15:11:57.780992 20251 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:11:57.863519 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 20054
2026-08-09T15:11:57Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64537 ms)
[----------] 30 tests from SecurityITest (296514 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:11:57 dist-test-slave-nj21 krb5kdc[20341](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:11:57 dist-test-slave-nj21 krb5kdc[20341](info): set up 2 sockets
Aug 09 15:11:57 dist-test-slave-nj21 krb5kdc[20341](info): commencing operation
krb5kdc: starting...
W20260809 15:11:59.096876 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.003s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:11:59 dist-test-slave-nj21 krb5kdc[20341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:11:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:11:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
I20260809 15:11:59.378646 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:45361
--webserver_interface=127.24.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:33901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:45361
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:11:59.710433 20359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:11:59.710705 20359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:11:59.710841 20359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:11:59.717872 20359 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:11:59.717981 20359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:11:59.718061 20359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:11:59.718138 20359 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:11:59.718199 20359 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:11:59.727936 20359 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:33901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:45361
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:45361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:11:59.729149 20359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:11:59.730450 20359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:11:59.742005 20378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:11:59.741973 20374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:11:59.742080 20373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:11:59.742791 20359 server_base.cc:1034] running on GCE node
I20260809 15:11:59.743505 20359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:11:59.744846 20359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:11:59.746057 20359 hybrid_clock.cc:648] HybridClock initialized: now 1786288319746008 us; error 27 us; skew 500 ppm
I20260809 15:11:59.746376 20359 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:11:59 dist-test-slave-nj21 krb5kdc[20341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288319, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:11:59.749385 20359 init.cc:377] Logged in from keytab as kudu/127.24.183.254@KRBTEST.COM (short username kudu)
I20260809 15:11:59.750876 20359 webserver.cc:460] Webserver started at http://127.24.183.254:33147/ using document root <none> and password file <none>
I20260809 15:11:59.751539 20359 fs_manager.cc:359] Metadata directory not provided
I20260809 15:11:59.751703 20359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:11:59.752071 20359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:11:59.755172 20359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "10a5a952feb4482382cbfd43707ea59a"
format_stamp: "Formatted at 2026-08-09 15:11:59 on dist-test-slave-nj21"
server_key: "08e85198950f30d1605d37a59ef52aab"
server_key_iv: "708287a0b74db93cc5be62db566c8930"
server_key_version: "encryptionkey@0"
I20260809 15:11:59.755995 20359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "10a5a952feb4482382cbfd43707ea59a"
format_stamp: "Formatted at 2026-08-09 15:11:59 on dist-test-slave-nj21"
server_key: "08e85198950f30d1605d37a59ef52aab"
server_key_iv: "708287a0b74db93cc5be62db566c8930"
server_key_version: "encryptionkey@0"
I20260809 15:11:59.760991 20359 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 15:11:59.764326 20390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:11:59.765545 20359 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:11:59.765705 20359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "10a5a952feb4482382cbfd43707ea59a"
format_stamp: "Formatted at 2026-08-09 15:11:59 on dist-test-slave-nj21"
server_key: "08e85198950f30d1605d37a59ef52aab"
server_key_iv: "708287a0b74db93cc5be62db566c8930"
server_key_version: "encryptionkey@0"
I20260809 15:11:59.765902 20359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:11:59.795543 20359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:11:59.799902 20359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:11:59.800175 20359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:11:59.818100 20359 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:45361
I20260809 15:11:59.818121 20477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:45361 every 8 connection(s)
I20260809 15:11:59.819438 20359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:11:59.823081 20478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:11:59.823377 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 20359
I20260809 15:11:59.823558 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:11:59.824244 25311 external_mini_cluster.cc:1428] Setting key 22c27bb2bf251afb4a771d8fb4df0081
I20260809 15:11:59.836462 20478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10a5a952feb4482382cbfd43707ea59a: Bootstrap starting.
Aug 09 15:11:59 dist-test-slave-nj21 krb5kdc[20341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:11:59.840193 20478 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10a5a952feb4482382cbfd43707ea59a: Neither blocks nor log segments found. Creating new log.
I20260809 15:11:59.841373 20478 log.cc:824] T 00000000000000000000000000000000 P 10a5a952feb4482382cbfd43707ea59a: Log is configured to *not* fsync() on all Append() calls
I20260809 15:11:59.844324 20478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10a5a952feb4482382cbfd43707ea59a: No bootstrap required, opened a new log
I20260809 15:11:59.847513 20486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:11:59.826161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35312 (local address 127.24.183.254:45361)
0809 15:11:59.826942 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0809 15:11:59.826953 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:11:59.826997 (+ 44us) server_negotiation.cc:407] Connection header received
0809 15:11:59.828994 (+ 1997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:11:59.829018 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:11:59.829377 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:11:59.829714 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:11:59.831458 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:59.832819 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:59.834736 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:11:59.835400 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:11:59.835524 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:11:59.840483 (+ 4959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:11:59.840491 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:11:59.840508 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:11:59.840546 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:11:59.843461 (+ 2915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:11:59.844288 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:11:59.844294 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:11:59.844300 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:11:59.844491 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:11:59.844905 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:11:59.844908 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:11:59.844910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:11:59.845440 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:11:59.845657 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0809 15:11:59.846131 (+ 474us) server_negotiation.cc:299] Negotiation successful
0809 15:11:59.846480 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":344,"threads_started":1}
I20260809 15:11:59.852883 20478 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10a5a952feb4482382cbfd43707ea59a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10a5a952feb4482382cbfd43707ea59a" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45361 } }
I20260809 15:11:59.853231 20478 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10a5a952feb4482382cbfd43707ea59a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:11:59.853318 20478 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10a5a952feb4482382cbfd43707ea59a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10a5a952feb4482382cbfd43707ea59a, State: Initialized, Role: FOLLOWER
I20260809 15:11:59.853835 20478 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10a5a952feb4482382cbfd43707ea59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10a5a952feb4482382cbfd43707ea59a" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45361 } }
I20260809 15:11:59.854069 20478 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10a5a952feb4482382cbfd43707ea59a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:11:59.854203 20478 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10a5a952feb4482382cbfd43707ea59a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:11:59.854379 20478 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10a5a952feb4482382cbfd43707ea59a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:11:59.856529 20478 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10a5a952feb4482382cbfd43707ea59a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10a5a952feb4482382cbfd43707ea59a" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45361 } }
I20260809 15:11:59.856985 20478 leader_election.cc:302] T 00000000000000000000000000000000 P 10a5a952feb4482382cbfd43707ea59a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10a5a952feb4482382cbfd43707ea59a; no voters:
I20260809 15:11:59.857511 20478 leader_election.cc:288] T 00000000000000000000000000000000 P 10a5a952feb4482382cbfd43707ea59a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:11:59.857635 20488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10a5a952feb4482382cbfd43707ea59a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:11:59.858356 20488 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10a5a952feb4482382cbfd43707ea59a [term 1 LEADER]: Becoming Leader. State: Replica: 10a5a952feb4482382cbfd43707ea59a, State: Running, Role: LEADER
I20260809 15:11:59.858865 20488 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10a5a952feb4482382cbfd43707ea59a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10a5a952feb4482382cbfd43707ea59a" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45361 } }
I20260809 15:11:59.859395 20478 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10a5a952feb4482382cbfd43707ea59a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:11:59.862179 20490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10a5a952feb4482382cbfd43707ea59a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10a5a952feb4482382cbfd43707ea59a. Latest consensus state: current_term: 1 leader_uuid: "10a5a952feb4482382cbfd43707ea59a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10a5a952feb4482382cbfd43707ea59a" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45361 } } }
I20260809 15:11:59.862538 20490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10a5a952feb4482382cbfd43707ea59a [sys.catalog]: This master's current role is: LEADER
I20260809 15:11:59.863881 20489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10a5a952feb4482382cbfd43707ea59a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10a5a952feb4482382cbfd43707ea59a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10a5a952feb4482382cbfd43707ea59a" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 45361 } } }
I20260809 15:11:59.864193 20489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10a5a952feb4482382cbfd43707ea59a [sys.catalog]: This master's current role is: LEADER
I20260809 15:11:59.864760 20504 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:11:59.869798 20504 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:11:59.878417 20504 catalog_manager.cc:1347] Generated new cluster ID: 85de9538c47f43b994b29b7ba8095d04
I20260809 15:11:59.878549 20504 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:11:59.886938 20504 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:11:59.888684 20504 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:11:59.903170 20504 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10a5a952feb4482382cbfd43707ea59a: Generated new TSK 0
I20260809 15:11:59.904160 20504 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab.
Entry for principal kudu/127.24.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab.
I20260809 15:11:59.973418 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.193:0
--local_ip_for_outbound_sockets=127.24.183.193
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:45361
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:33901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:12:00.312541 20522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:00.312805 20522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:00.312937 20522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:00.320036 20522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:00.320134 20522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:00.320292 20522 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.193
I20260809 15:12:00.329700 20522 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:33901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:45361
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.193
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:00.330967 20522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:00.332321 20522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:00.344836 20527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:00.344877 20530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:00.344913 20528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:00.345635 20522 server_base.cc:1034] running on GCE node
I20260809 15:12:00.346205 20522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:00.347090 20522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:00.348287 20522 hybrid_clock.cc:648] HybridClock initialized: now 1786288320348242 us; error 27 us; skew 500 ppm
I20260809 15:12:00.348580 20522 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:12:00 dist-test-slave-nj21 krb5kdc[20341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288320, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:12:00.351716 20522 init.cc:377] Logged in from keytab as kudu/127.24.183.193@KRBTEST.COM (short username kudu)
I20260809 15:12:00.353075 20522 webserver.cc:460] Webserver started at http://127.24.183.193:34271/ using document root <none> and password file <none>
I20260809 15:12:00.353703 20522 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:00.353827 20522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:00.354120 20522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:00.357017 20522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-0/data/instance:
uuid: "6b23c2a59c3a4078b5f970e0a70bfc64"
format_stamp: "Formatted at 2026-08-09 15:12:00 on dist-test-slave-nj21"
server_key: "6f436e893a87c0236ed563a7638a7e9b"
server_key_iv: "7aac08c897534e069781886efe7f0b03"
server_key_version: "encryptionkey@0"
I20260809 15:12:00.357752 20522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance:
uuid: "6b23c2a59c3a4078b5f970e0a70bfc64"
format_stamp: "Formatted at 2026-08-09 15:12:00 on dist-test-slave-nj21"
server_key: "6f436e893a87c0236ed563a7638a7e9b"
server_key_iv: "7aac08c897534e069781886efe7f0b03"
server_key_version: "encryptionkey@0"
I20260809 15:12:00.362699 20522 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260809 15:12:00.365960 20539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:00.367200 20522 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:12:00.367349 20522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-0/wal
uuid: "6b23c2a59c3a4078b5f970e0a70bfc64"
format_stamp: "Formatted at 2026-08-09 15:12:00 on dist-test-slave-nj21"
server_key: "6f436e893a87c0236ed563a7638a7e9b"
server_key_iv: "7aac08c897534e069781886efe7f0b03"
server_key_version: "encryptionkey@0"
I20260809 15:12:00.367517 20522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:00.398912 20522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:00.402748 20522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:00.403034 20522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:00.404109 20522 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:12:00.405432 20522 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:12:00.405514 20522 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:00.405655 20522 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:12:00.405707 20522 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:00.436168 20522 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.193:34025
I20260809 15:12:00.436220 20733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.193:34025 every 8 connection(s)
I20260809 15:12:00.437423 20522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
I20260809 15:12:00.440445 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 20522
I20260809 15:12:00.440672 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance
I20260809 15:12:00.441632 25311 external_mini_cluster.cc:1428] Setting key 456944a310adea0944ff498d49a054b1
Aug 09 15:12:00 dist-test-slave-nj21 krb5kdc[20341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288320, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:12:00.458777 20738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:00.439904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:49179 (local address 127.24.183.254:45361)
0809 15:12:00.440442 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:00.440448 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:00.441472 (+ 1024us) server_negotiation.cc:407] Connection header received
0809 15:12:00.443813 (+ 2341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:00.443817 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:00.443881 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:00.443969 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:00.445983 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:00.446713 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:00.448882 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:00.449288 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:00.449381 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:00.453273 (+ 3892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:00.453278 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:00.453281 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:12:00.453315 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:00.455827 (+ 2512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:00.456502 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:00.456506 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:00.456508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:00.456625 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:00.457078 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:00.457083 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:00.457085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:00.457350 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:00.457496 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:00.458182 (+ 686us) server_negotiation.cc:299] Negotiation successful
0809 15:12:00.458418 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":266,"threads_started":1}
I20260809 15:12:00.459779 20743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:00.440351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45361 (local address 127.24.183.193:49179)
0809 15:12:00.441287 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:00.441342 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:00.443511 (+ 2169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:00.444250 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:00.444262 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:00.445017 (+ 755us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:12:00.445814 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:00.445840 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:00.446889 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:00.446894 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:00.448753 (+ 1859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:00.448768 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:00.449538 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:00.449546 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:00.449833 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:00.450501 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:12:00.450532 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:12:00.453059 (+ 2527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:12:00.455952 (+ 2893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:12:00.455961 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:12:00.455977 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:12:00.456378 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:12:00.456724 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:12:00.456729 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:12:00.456733 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:12:00.456924 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:12:00.457480 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:12:00.457488 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:12:00.457806 (+ 318us) client_negotiation.cc:769] Sending connection context
0809 15:12:00.458044 (+ 238us) client_negotiation.cc:240] Negotiation successful
0809 15:12:00.458539 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":359,"threads_started":1}
I20260809 15:12:00.461336 20734 heartbeater.cc:347] Connected to a master server at 127.24.183.254:45361
I20260809 15:12:00.461616 20734 heartbeater.cc:464] Registering TS with master...
I20260809 15:12:00.462304 20734 heartbeater.cc:511] Master 127.24.183.254:45361 requested a full tablet report, sending...
I20260809 15:12:00.464681 20415 ts_manager.cc:194] Registered new tserver with Master: 6b23c2a59c3a4078b5f970e0a70bfc64 (127.24.183.193:34025)
I20260809 15:12:00.466807 20415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.193@KRBTEST.COM'} at 127.24.183.193:49179
WARNING: no policy specified for kudu/127.24.183.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab.
Entry for principal kudu/127.24.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab.
I20260809 15:12:00.510681 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.194:0
--local_ip_for_outbound_sockets=127.24.183.194
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:45361
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:33901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:12:00.843986 20748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:00.844254 20748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:00.844357 20748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:00.851504 20748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:00.851606 20748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:00.851775 20748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.194
I20260809 15:12:00.861717 20748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:33901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:45361
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.194
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:00.862926 20748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:00.864312 20748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:00.877264 20759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:00.877038 20754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:00.877480 20753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:00.878728 20748 server_base.cc:1034] running on GCE node
I20260809 15:12:00.879328 20748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:00.880254 20748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:00.881460 20748 hybrid_clock.cc:648] HybridClock initialized: now 1786288320881411 us; error 36 us; skew 500 ppm
I20260809 15:12:00.881758 20748 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:12:00 dist-test-slave-nj21 krb5kdc[20341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288320, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:12:00.890400 20748 init.cc:377] Logged in from keytab as kudu/127.24.183.194@KRBTEST.COM (short username kudu)
I20260809 15:12:00.891922 20748 webserver.cc:460] Webserver started at http://127.24.183.194:43505/ using document root <none> and password file <none>
I20260809 15:12:00.892622 20748 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:00.892740 20748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:00.893052 20748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:00.896075 20748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-1/data/instance:
uuid: "fc11ee6ea097442f9cbe5aaccc832e25"
format_stamp: "Formatted at 2026-08-09 15:12:00 on dist-test-slave-nj21"
server_key: "fcca7c47a722b09d961c46adc86bb67d"
server_key_iv: "9cbea7138a2d0d06e504a01a383072d2"
server_key_version: "encryptionkey@0"
I20260809 15:12:00.896827 20748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance:
uuid: "fc11ee6ea097442f9cbe5aaccc832e25"
format_stamp: "Formatted at 2026-08-09 15:12:00 on dist-test-slave-nj21"
server_key: "fcca7c47a722b09d961c46adc86bb67d"
server_key_iv: "9cbea7138a2d0d06e504a01a383072d2"
server_key_version: "encryptionkey@0"
I20260809 15:12:00.901948 20748 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 15:12:00.905300 20773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:00.906519 20748 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 15:12:00.906662 20748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-1/wal
uuid: "fc11ee6ea097442f9cbe5aaccc832e25"
format_stamp: "Formatted at 2026-08-09 15:12:00 on dist-test-slave-nj21"
server_key: "fcca7c47a722b09d961c46adc86bb67d"
server_key_iv: "9cbea7138a2d0d06e504a01a383072d2"
server_key_version: "encryptionkey@0"
I20260809 15:12:00.906837 20748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:00.943275 20748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:00.947846 20748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:00.948292 20748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:00.949348 20748 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:12:00.950989 20748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:12:00.951077 20748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:00.951207 20748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:12:00.951268 20748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:00.984720 20748 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.194:34367
I20260809 15:12:00.984822 20969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.194:34367 every 8 connection(s)
I20260809 15:12:00.986088 20748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
I20260809 15:12:00.989641 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 20748
I20260809 15:12:00.989820 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance
I20260809 15:12:00.990432 25311 external_mini_cluster.cc:1428] Setting key d6e0566d8d089ab7bc366c87e2419c57
Aug 09 15:12:00 dist-test-slave-nj21 krb5kdc[20341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288320, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:12:01.012203 20974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:00.988884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:44271 (local address 127.24.183.254:45361)
0809 15:12:00.989667 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:00.989674 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:00.990915 (+ 1241us) server_negotiation.cc:407] Connection header received
0809 15:12:00.993277 (+ 2362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:00.993281 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:00.993361 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:00.993479 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:00.996137 (+ 2658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:00.996909 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:00.999145 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:00.999620 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:00.999831 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:01.004170 (+ 4339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:01.004176 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:01.004180 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:12:01.004226 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:01.008186 (+ 3960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:01.009206 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:01.009212 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:01.009215 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:01.009400 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:01.009971 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:01.009975 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:01.009978 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:01.010352 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:01.010518 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:01.011516 (+ 998us) server_negotiation.cc:299] Negotiation successful
0809 15:12:01.011854 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":370,"threads_started":1}
I20260809 15:12:01.013108 20981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:00.989785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45361 (local address 127.24.183.194:44271)
0809 15:12:00.990693 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:00.990767 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:00.992941 (+ 2174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:00.993809 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:00.993821 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:00.994670 (+ 849us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:12:00.995908 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:00.995939 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:00.997079 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:00.997084 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:00.998970 (+ 1886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:00.998986 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:00.999929 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:00.999937 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:01.000227 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:01.001011 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:12:01.001046 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:12:01.003890 (+ 2844us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:12:01.008374 (+ 4484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:12:01.008385 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:12:01.008405 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:12:01.009044 (+ 639us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:12:01.009557 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:12:01.009563 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:12:01.009566 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:12:01.009844 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:12:01.010513 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:12:01.010522 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:12:01.011060 (+ 538us) client_negotiation.cc:769] Sending connection context
0809 15:12:01.011355 (+ 295us) client_negotiation.cc:240] Negotiation successful
0809 15:12:01.011880 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":331,"threads_started":1}
I20260809 15:12:01.014619 20970 heartbeater.cc:347] Connected to a master server at 127.24.183.254:45361
I20260809 15:12:01.015093 20970 heartbeater.cc:464] Registering TS with master...
I20260809 15:12:01.015980 20970 heartbeater.cc:511] Master 127.24.183.254:45361 requested a full tablet report, sending...
I20260809 15:12:01.017621 20415 ts_manager.cc:194] Registered new tserver with Master: fc11ee6ea097442f9cbe5aaccc832e25 (127.24.183.194:34367)
I20260809 15:12:01.018479 20415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.194@KRBTEST.COM'} at 127.24.183.194:44271
WARNING: no policy specified for kudu/127.24.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab.
Entry for principal kudu/127.24.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab.
I20260809 15:12:01.065035 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.195:0
--local_ip_for_outbound_sockets=127.24.183.195
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:45361
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:33901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:12:01.409161 20986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:01.409466 20986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:01.409610 20986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:01.416971 20986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:01.417071 20986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:01.417235 20986 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.195
I20260809 15:12:01.427006 20986 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:33901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:45361
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.195
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:01.428313 20986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:01.429615 20986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:01.442405 20993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:01.442441 20995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:01.442574 20997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:01.443559 20986 server_base.cc:1034] running on GCE node
I20260809 15:12:01.444130 20986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:01.445010 20986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:01.446204 20986 hybrid_clock.cc:648] HybridClock initialized: now 1786288321446153 us; error 43 us; skew 500 ppm
I20260809 15:12:01.446506 20986 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:12:01 dist-test-slave-nj21 krb5kdc[20341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288321, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:12:01.449749 20986 init.cc:377] Logged in from keytab as kudu/127.24.183.195@KRBTEST.COM (short username kudu)
I20260809 15:12:01.451203 20986 webserver.cc:460] Webserver started at http://127.24.183.195:33699/ using document root <none> and password file <none>
I20260809 15:12:01.451926 20986 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:01.452046 20986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:01.452360 20986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:01.455348 20986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-2/data/instance:
uuid: "bbd73fc78a084ab5814444308af8da41"
format_stamp: "Formatted at 2026-08-09 15:12:01 on dist-test-slave-nj21"
server_key: "476e49a19b9a37f3842daacadffc502e"
server_key_iv: "19d8b85d307b2cd2ce06aca8c0cd3f28"
server_key_version: "encryptionkey@0"
I20260809 15:12:01.456142 20986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance:
uuid: "bbd73fc78a084ab5814444308af8da41"
format_stamp: "Formatted at 2026-08-09 15:12:01 on dist-test-slave-nj21"
server_key: "476e49a19b9a37f3842daacadffc502e"
server_key_iv: "19d8b85d307b2cd2ce06aca8c0cd3f28"
server_key_version: "encryptionkey@0"
I20260809 15:12:01.461257 20986 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 15:12:01.464862 21012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:01.466274 20986 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:12:01.466444 20986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-2/wal
uuid: "bbd73fc78a084ab5814444308af8da41"
format_stamp: "Formatted at 2026-08-09 15:12:01 on dist-test-slave-nj21"
server_key: "476e49a19b9a37f3842daacadffc502e"
server_key_iv: "19d8b85d307b2cd2ce06aca8c0cd3f28"
server_key_version: "encryptionkey@0"
I20260809 15:12:01.466632 20986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:01.469858 20734 heartbeater.cc:503] Master 127.24.183.254:45361 was elected leader, sending a full tablet report...
I20260809 15:12:01.482321 20986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:01.486340 20986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:01.486610 20986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:01.487574 20986 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:12:01.488981 20986 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:12:01.489066 20986 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:01.489199 20986 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:12:01.489260 20986 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:01.520946 20986 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.195:39665
I20260809 15:12:01.520995 21207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.195:39665 every 8 connection(s)
I20260809 15:12:01.522251 20986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
I20260809 15:12:01.525915 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 20986
I20260809 15:12:01.526111 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance
I20260809 15:12:01.526652 25311 external_mini_cluster.cc:1428] Setting key 6d44638bb1b01dd9ae0780e0f5d67a04
Aug 09 15:12:01 dist-test-slave-nj21 krb5kdc[20341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288321, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:12:01.541949 21212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:01.524809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:37641 (local address 127.24.183.254:45361)
0809 15:12:01.525328 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:01.525334 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:01.526262 (+ 928us) server_negotiation.cc:407] Connection header received
0809 15:12:01.528367 (+ 2105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:01.528373 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:01.528458 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:01.528564 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:01.530143 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:01.531018 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:01.532619 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:01.533036 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:01.533139 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:01.536439 (+ 3300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:01.536443 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:01.536447 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:12:01.536481 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:01.538945 (+ 2464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:01.539602 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:01.539606 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:01.539608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:01.539775 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:01.540213 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:01.540217 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:01.540219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:01.540577 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:01.540737 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:01.541431 (+ 694us) server_negotiation.cc:299] Negotiation successful
0809 15:12:01.541645 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":229,"threads_started":1}
I20260809 15:12:01.542971 21213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:01.525242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:45361 (local address 127.24.183.195:37641)
0809 15:12:01.526044 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:01.526095 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:01.528098 (+ 2003us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:01.528784 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:01.528792 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:01.529313 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:12:01.529985 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:01.530005 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:01.531182 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:01.531186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:01.532463 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:01.532476 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:01.533215 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:01.533220 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:01.533437 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:01.534056 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:12:01.534080 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:12:01.536243 (+ 2163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:12:01.539073 (+ 2830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:12:01.539080 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:12:01.539091 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:12:01.539510 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:12:01.539888 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:12:01.539892 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:12:01.539895 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:12:01.540081 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:12:01.540709 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:12:01.540717 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:12:01.541039 (+ 322us) client_negotiation.cc:769] Sending connection context
0809 15:12:01.541288 (+ 249us) client_negotiation.cc:240] Negotiation successful
0809 15:12:01.541815 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":307,"threads_started":1}
I20260809 15:12:01.544345 21208 heartbeater.cc:347] Connected to a master server at 127.24.183.254:45361
I20260809 15:12:01.544639 21208 heartbeater.cc:464] Registering TS with master...
I20260809 15:12:01.545301 21208 heartbeater.cc:511] Master 127.24.183.254:45361 requested a full tablet report, sending...
I20260809 15:12:01.546686 20415 ts_manager.cc:194] Registered new tserver with Master: bbd73fc78a084ab5814444308af8da41 (127.24.183.195:39665)
I20260809 15:12:01.547458 20415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.195@KRBTEST.COM'} at 127.24.183.195:37641
I20260809 15:12:01.554471 25311 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:12:01.571956 21212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:01.559265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35324 (local address 127.24.183.254:45361)
0809 15:12:01.559423 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:01.559428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:01.559907 (+ 479us) server_negotiation.cc:407] Connection header received
0809 15:12:01.560075 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:01.560080 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:01.560169 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:01.560263 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:01.561646 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:01.562399 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:01.563745 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:01.564165 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:01.564256 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:01.566514 (+ 2258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:01.566519 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:01.566522 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:12:01.566554 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:01.569228 (+ 2674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:01.569918 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:01.569922 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:01.569924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:01.570071 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:01.570449 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:01.570453 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:01.570473 (+ 20us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:01.570885 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:01.571148 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:01.571440 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 15:12:01.571648 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 15:12:01.576035 20415 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35324:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:12:01.583738 20415 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:12:01.606148 21235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:01.596626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47324 (local address 127.24.183.194:34367)
0809 15:12:01.597211 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:01.597221 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:01.597258 (+ 37us) server_negotiation.cc:407] Connection header received
0809 15:12:01.597388 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:01.597396 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:01.597599 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:01.597797 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:12:01.598610 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:01.600242 (+ 1632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:01.603310 (+ 3068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:01.604768 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:01.604949 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:01.605148 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:01.605514 (+ 366us) server_negotiation.cc:299] Negotiation successful
0809 15:12:01.605713 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":681,"threads_started":1}
I20260809 15:12:01.606343 21232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:01.596191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:34367 (local address 127.0.0.1:47324)
0809 15:12:01.596899 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:01.596944 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:01.597139 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:01.597839 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:01.597848 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:01.597883 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:12:01.598329 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:01.598345 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:01.600419 (+ 2074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:01.600425 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:01.603086 (+ 2661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:01.603110 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:01.605024 (+ 1914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:01.605033 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:01.605270 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:01.605318 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 15:12:01.605442 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 15:12:01.605648 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":1326,"threads_started":1}
I20260809 15:12:01.609620 21241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:01.599150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43808 (local address 127.24.183.195:39665)
0809 15:12:01.600585 (+ 1435us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:01.600594 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:01.601241 (+ 647us) server_negotiation.cc:407] Connection header received
0809 15:12:01.601473 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:01.601481 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:01.601653 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:01.601822 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:12:01.602647 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:01.603916 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:01.607241 (+ 3325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:01.608407 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:01.608615 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:01.608793 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:01.609052 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 15:12:01.609229 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1262,"thread_start_us":1064,"threads_started":1}
I20260809 15:12:01.609656 21234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:01.596405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:34025 (local address 127.0.0.1:49004)
0809 15:12:01.600459 (+ 4054us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:01.600481 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:01.600713 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:01.601497 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:01.601502 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:01.601529 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:12:01.602370 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:01.602387 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:01.604439 (+ 2052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:01.604448 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:01.606793 (+ 2345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:01.606807 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:01.608786 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:01.608794 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:01.608957 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:01.608988 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 15:12:01.609075 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 15:12:01.609196 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2301,"mutex_wait_us":19,"thread_start_us":2461,"threads_started":1}
I20260809 15:12:01.609720 21239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:01.598650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:39665 (local address 127.0.0.1:43808)
0809 15:12:01.601108 (+ 2458us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:01.601127 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:01.601286 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:01.601827 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:01.601832 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:01.601860 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:12:01.602384 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:01.602398 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:01.604047 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:01.604052 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:01.607068 (+ 3016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:01.607082 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:01.608614 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:01.608621 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:01.608842 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:01.608882 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 15:12:01.608970 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 15:12:01.609109 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2333,"mutex_wait_us":55,"thread_start_us":2126,"threads_started":1}
I20260809 15:12:01.610373 21238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:01.597118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49004 (local address 127.24.183.193:34025)
0809 15:12:01.598574 (+ 1456us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:01.598583 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:01.600589 (+ 2006us) server_negotiation.cc:407] Connection header received
0809 15:12:01.600907 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:01.600915 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:01.601138 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:01.601300 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:12:01.602649 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:01.604302 (+ 1653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:01.606968 (+ 2666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:01.608226 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:01.608415 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:01.608576 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:01.609882 (+ 1306us) server_negotiation.cc:299] Negotiation successful
0809 15:12:01.610049 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1273,"thread_start_us":1078,"threads_started":1}
I20260809 15:12:01.611275 20867 tablet_service.cc:1467] Processing CreateTablet for tablet 05ac8774377744f99a97d9bd97368b5a (DEFAULT_TABLE table=test-table [id=be93960c17434759b9a0cfb620d6ad83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:12:01.612780 20867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05ac8774377744f99a97d9bd97368b5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:01.614506 21103 tablet_service.cc:1467] Processing CreateTablet for tablet 05ac8774377744f99a97d9bd97368b5a (DEFAULT_TABLE table=test-table [id=be93960c17434759b9a0cfb620d6ad83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:12:01.614501 20624 tablet_service.cc:1467] Processing CreateTablet for tablet 05ac8774377744f99a97d9bd97368b5a (DEFAULT_TABLE table=test-table [id=be93960c17434759b9a0cfb620d6ad83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:12:01.616107 21103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05ac8774377744f99a97d9bd97368b5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:01.616678 20624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05ac8774377744f99a97d9bd97368b5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:01.624377 21243 tablet_bootstrap.cc:492] T 05ac8774377744f99a97d9bd97368b5a P fc11ee6ea097442f9cbe5aaccc832e25: Bootstrap starting.
I20260809 15:12:01.627504 21243 tablet_bootstrap.cc:654] T 05ac8774377744f99a97d9bd97368b5a P fc11ee6ea097442f9cbe5aaccc832e25: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:01.628562 21243 log.cc:824] T 05ac8774377744f99a97d9bd97368b5a P fc11ee6ea097442f9cbe5aaccc832e25: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:01.631645 21248 tablet_bootstrap.cc:492] T 05ac8774377744f99a97d9bd97368b5a P bbd73fc78a084ab5814444308af8da41: Bootstrap starting.
I20260809 15:12:01.632486 21247 tablet_bootstrap.cc:492] T 05ac8774377744f99a97d9bd97368b5a P 6b23c2a59c3a4078b5f970e0a70bfc64: Bootstrap starting.
I20260809 15:12:01.633373 21243 tablet_bootstrap.cc:492] T 05ac8774377744f99a97d9bd97368b5a P fc11ee6ea097442f9cbe5aaccc832e25: No bootstrap required, opened a new log
I20260809 15:12:01.633661 21243 ts_tablet_manager.cc:1397] T 05ac8774377744f99a97d9bd97368b5a P fc11ee6ea097442f9cbe5aaccc832e25: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260809 15:12:01.636147 21247 tablet_bootstrap.cc:654] T 05ac8774377744f99a97d9bd97368b5a P 6b23c2a59c3a4078b5f970e0a70bfc64: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:01.637405 21247 log.cc:824] T 05ac8774377744f99a97d9bd97368b5a P 6b23c2a59c3a4078b5f970e0a70bfc64: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:01.639081 21248 tablet_bootstrap.cc:654] T 05ac8774377744f99a97d9bd97368b5a P bbd73fc78a084ab5814444308af8da41: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:01.640228 21248 log.cc:824] T 05ac8774377744f99a97d9bd97368b5a P bbd73fc78a084ab5814444308af8da41: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:01.642510 21247 tablet_bootstrap.cc:492] T 05ac8774377744f99a97d9bd97368b5a P 6b23c2a59c3a4078b5f970e0a70bfc64: No bootstrap required, opened a new log
I20260809 15:12:01.642719 21248 tablet_bootstrap.cc:492] T 05ac8774377744f99a97d9bd97368b5a P bbd73fc78a084ab5814444308af8da41: No bootstrap required, opened a new log
I20260809 15:12:01.642802 21247 ts_tablet_manager.cc:1397] T 05ac8774377744f99a97d9bd97368b5a P 6b23c2a59c3a4078b5f970e0a70bfc64: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260809 15:12:01.642975 21248 ts_tablet_manager.cc:1397] T 05ac8774377744f99a97d9bd97368b5a P bbd73fc78a084ab5814444308af8da41: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.003s
I20260809 15:12:01.643919 21243 raft_consensus.cc:348] T 05ac8774377744f99a97d9bd97368b5a P fc11ee6ea097442f9cbe5aaccc832e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11ee6ea097442f9cbe5aaccc832e25" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 34367 } } peers { permanent_uuid: "6b23c2a59c3a4078b5f970e0a70bfc64" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 34025 } } peers { permanent_uuid: "bbd73fc78a084ab5814444308af8da41" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 39665 } }
I20260809 15:12:01.644416 21243 raft_consensus.cc:374] T 05ac8774377744f99a97d9bd97368b5a P fc11ee6ea097442f9cbe5aaccc832e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:01.644518 21243 raft_consensus.cc:729] T 05ac8774377744f99a97d9bd97368b5a P fc11ee6ea097442f9cbe5aaccc832e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc11ee6ea097442f9cbe5aaccc832e25, State: Initialized, Role: FOLLOWER
I20260809 15:12:01.645144 21243 consensus_queue.cc:260] T 05ac8774377744f99a97d9bd97368b5a P fc11ee6ea097442f9cbe5aaccc832e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11ee6ea097442f9cbe5aaccc832e25" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 34367 } } peers { permanent_uuid: "6b23c2a59c3a4078b5f970e0a70bfc64" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 34025 } } peers { permanent_uuid: "bbd73fc78a084ab5814444308af8da41" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 39665 } }
I20260809 15:12:01.646309 20970 heartbeater.cc:503] Master 127.24.183.254:45361 was elected leader, sending a full tablet report...
I20260809 15:12:01.646950 21243 ts_tablet_manager.cc:1428] T 05ac8774377744f99a97d9bd97368b5a P fc11ee6ea097442f9cbe5aaccc832e25: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260809 15:12:01.651885 21248 raft_consensus.cc:348] T 05ac8774377744f99a97d9bd97368b5a P bbd73fc78a084ab5814444308af8da41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11ee6ea097442f9cbe5aaccc832e25" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 34367 } } peers { permanent_uuid: "6b23c2a59c3a4078b5f970e0a70bfc64" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 34025 } } peers { permanent_uuid: "bbd73fc78a084ab5814444308af8da41" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 39665 } }
I20260809 15:12:01.652243 21248 raft_consensus.cc:374] T 05ac8774377744f99a97d9bd97368b5a P bbd73fc78a084ab5814444308af8da41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:01.652311 21248 raft_consensus.cc:729] T 05ac8774377744f99a97d9bd97368b5a P bbd73fc78a084ab5814444308af8da41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbd73fc78a084ab5814444308af8da41, State: Initialized, Role: FOLLOWER
I20260809 15:12:01.652186 21247 raft_consensus.cc:348] T 05ac8774377744f99a97d9bd97368b5a P 6b23c2a59c3a4078b5f970e0a70bfc64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11ee6ea097442f9cbe5aaccc832e25" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 34367 } } peers { permanent_uuid: "6b23c2a59c3a4078b5f970e0a70bfc64" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 34025 } } peers { permanent_uuid: "bbd73fc78a084ab5814444308af8da41" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 39665 } }
I20260809 15:12:01.652601 21247 raft_consensus.cc:374] T 05ac8774377744f99a97d9bd97368b5a P 6b23c2a59c3a4078b5f970e0a70bfc64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:01.652697 21247 raft_consensus.cc:729] T 05ac8774377744f99a97d9bd97368b5a P 6b23c2a59c3a4078b5f970e0a70bfc64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b23c2a59c3a4078b5f970e0a70bfc64, State: Initialized, Role: FOLLOWER
I20260809 15:12:01.652818 21248 consensus_queue.cc:260] T 05ac8774377744f99a97d9bd97368b5a P bbd73fc78a084ab5814444308af8da41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11ee6ea097442f9cbe5aaccc832e25" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 34367 } } peers { permanent_uuid: "6b23c2a59c3a4078b5f970e0a70bfc64" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 34025 } } peers { permanent_uuid: "bbd73fc78a084ab5814444308af8da41" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 39665 } }
I20260809 15:12:01.653273 21247 consensus_queue.cc:260] T 05ac8774377744f99a97d9bd97368b5a P 6b23c2a59c3a4078b5f970e0a70bfc64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11ee6ea097442f9cbe5aaccc832e25" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 34367 } } peers { permanent_uuid: "6b23c2a59c3a4078b5f970e0a70bfc64" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 34025 } } peers { permanent_uuid: "bbd73fc78a084ab5814444308af8da41" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 39665 } }
I20260809 15:12:01.653937 21208 heartbeater.cc:503] Master 127.24.183.254:45361 was elected leader, sending a full tablet report...
I20260809 15:12:01.654552 21248 ts_tablet_manager.cc:1428] T 05ac8774377744f99a97d9bd97368b5a P bbd73fc78a084ab5814444308af8da41: Time spent starting tablet: real 0.011s user 0.011s sys 0.001s
I20260809 15:12:01.655048 21247 ts_tablet_manager.cc:1428] T 05ac8774377744f99a97d9bd97368b5a P 6b23c2a59c3a4078b5f970e0a70bfc64: Time spent starting tablet: real 0.012s user 0.009s sys 0.004s
W20260809 15:12:01.689828 20735 tablet.cc:2369] T 05ac8774377744f99a97d9bd97368b5a P 6b23c2a59c3a4078b5f970e0a70bfc64: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:12:01.719283 21253 raft_consensus.cc:482] T 05ac8774377744f99a97d9bd97368b5a P bbd73fc78a084ab5814444308af8da41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:12:01.719566 21253 raft_consensus.cc:504] T 05ac8774377744f99a97d9bd97368b5a P bbd73fc78a084ab5814444308af8da41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11ee6ea097442f9cbe5aaccc832e25" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 34367 } } peers { permanent_uuid: "6b23c2a59c3a4078b5f970e0a70bfc64" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 34025 } } peers { permanent_uuid: "bbd73fc78a084ab5814444308af8da41" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 39665 } }
I20260809 15:12:01.721465 21253 leader_election.cc:288] T 05ac8774377744f99a97d9bd97368b5a P bbd73fc78a084ab5814444308af8da41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc11ee6ea097442f9cbe5aaccc832e25 (127.24.183.194:34367), 6b23c2a59c3a4078b5f970e0a70bfc64 (127.24.183.193:34025)
I20260809 15:12:01.728535 21235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:01.721936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:41717 (local address 127.24.183.194:34367)
0809 15:12:01.722141 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:01.722148 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:01.722602 (+ 454us) server_negotiation.cc:407] Connection header received
0809 15:12:01.722714 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:01.722718 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:01.722780 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:01.722850 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:12:01.723262 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:01.724246 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:01.726606 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:01.727755 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:01.727903 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:01.727980 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:01.728198 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 15:12:01.728330 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 15:12:01.728598 21213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:01.721813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:34367 (local address 127.24.183.195:41717)
0809 15:12:01.722412 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:01.722437 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:01.722584 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:01.722905 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:01.722910 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:01.722940 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:12:01.723128 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:01.723138 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:01.724400 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:01.724404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:01.726447 (+ 2043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:01.726467 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:01.727904 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:01.727909 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:01.728049 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:01.728076 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 15:12:01.728152 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 15:12:01.728301 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":32}
I20260809 15:12:01.728971 21238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:01.722216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:33167 (local address 127.24.183.193:34025)
0809 15:12:01.722448 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:01.722455 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:01.722964 (+ 509us) server_negotiation.cc:407] Connection header received
0809 15:12:01.723084 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:01.723088 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:01.723149 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:01.723227 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:12:01.723758 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:01.724648 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:01.726921 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:01.728093 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:01.728228 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:01.728293 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:01.728615 (+ 322us) server_negotiation.cc:299] Negotiation successful
0809 15:12:01.728740 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 15:12:01.729066 21256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:01.722044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:34025 (local address 127.24.183.195:33167)
0809 15:12:01.722834 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:01.722855 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:01.722964 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:01.723256 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:01.723260 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:01.723280 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:12:01.723571 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:01.723579 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:01.724821 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:01.724826 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:01.726759 (+ 1933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:01.726770 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:01.728279 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:01.728284 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:01.728450 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:01.728483 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 15:12:01.728568 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 15:12:01.728678 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"mutex_wait_us":4,"thread_start_us":419,"threads_started":1}
I20260809 15:12:01.729641 20903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05ac8774377744f99a97d9bd97368b5a" candidate_uuid: "bbd73fc78a084ab5814444308af8da41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc11ee6ea097442f9cbe5aaccc832e25" is_pre_election: true
I20260809 15:12:01.730099 20903 raft_consensus.cc:2454] T 05ac8774377744f99a97d9bd97368b5a P fc11ee6ea097442f9cbe5aaccc832e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbd73fc78a084ab5814444308af8da41 in term 0.
I20260809 15:12:01.730014 20660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05ac8774377744f99a97d9bd97368b5a" candidate_uuid: "bbd73fc78a084ab5814444308af8da41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b23c2a59c3a4078b5f970e0a70bfc64" is_pre_election: true
I20260809 15:12:01.730458 20660 raft_consensus.cc:2454] T 05ac8774377744f99a97d9bd97368b5a P 6b23c2a59c3a4078b5f970e0a70bfc64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbd73fc78a084ab5814444308af8da41 in term 0.
I20260809 15:12:01.730764 21014 leader_election.cc:302] T 05ac8774377744f99a97d9bd97368b5a P bbd73fc78a084ab5814444308af8da41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bbd73fc78a084ab5814444308af8da41, fc11ee6ea097442f9cbe5aaccc832e25; no voters:
I20260809 15:12:01.731199 21253 raft_consensus.cc:2781] T 05ac8774377744f99a97d9bd97368b5a P bbd73fc78a084ab5814444308af8da41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:12:01.731335 21253 raft_consensus.cc:482] T 05ac8774377744f99a97d9bd97368b5a P bbd73fc78a084ab5814444308af8da41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:12:01.731431 21253 raft_consensus.cc:3037] T 05ac8774377744f99a97d9bd97368b5a P bbd73fc78a084ab5814444308af8da41 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:01.733700 21253 raft_consensus.cc:504] T 05ac8774377744f99a97d9bd97368b5a P bbd73fc78a084ab5814444308af8da41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11ee6ea097442f9cbe5aaccc832e25" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 34367 } } peers { permanent_uuid: "6b23c2a59c3a4078b5f970e0a70bfc64" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 34025 } } peers { permanent_uuid: "bbd73fc78a084ab5814444308af8da41" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 39665 } }
I20260809 15:12:01.734251 21253 leader_election.cc:288] T 05ac8774377744f99a97d9bd97368b5a P bbd73fc78a084ab5814444308af8da41 [CANDIDATE]: Term 1 election: Requested vote from peers fc11ee6ea097442f9cbe5aaccc832e25 (127.24.183.194:34367), 6b23c2a59c3a4078b5f970e0a70bfc64 (127.24.183.193:34025)
I20260809 15:12:01.734649 20903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05ac8774377744f99a97d9bd97368b5a" candidate_uuid: "bbd73fc78a084ab5814444308af8da41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc11ee6ea097442f9cbe5aaccc832e25"
I20260809 15:12:01.734771 20660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05ac8774377744f99a97d9bd97368b5a" candidate_uuid: "bbd73fc78a084ab5814444308af8da41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b23c2a59c3a4078b5f970e0a70bfc64"
I20260809 15:12:01.734917 20903 raft_consensus.cc:3037] T 05ac8774377744f99a97d9bd97368b5a P fc11ee6ea097442f9cbe5aaccc832e25 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:01.734967 20660 raft_consensus.cc:3037] T 05ac8774377744f99a97d9bd97368b5a P 6b23c2a59c3a4078b5f970e0a70bfc64 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:01.738238 20660 raft_consensus.cc:2454] T 05ac8774377744f99a97d9bd97368b5a P 6b23c2a59c3a4078b5f970e0a70bfc64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbd73fc78a084ab5814444308af8da41 in term 1.
I20260809 15:12:01.738283 20903 raft_consensus.cc:2454] T 05ac8774377744f99a97d9bd97368b5a P fc11ee6ea097442f9cbe5aaccc832e25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbd73fc78a084ab5814444308af8da41 in term 1.
W20260809 15:12:01.738319 20972 tablet.cc:2369] T 05ac8774377744f99a97d9bd97368b5a P fc11ee6ea097442f9cbe5aaccc832e25: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:12:01.738655 21014 leader_election.cc:302] T 05ac8774377744f99a97d9bd97368b5a P bbd73fc78a084ab5814444308af8da41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bbd73fc78a084ab5814444308af8da41, fc11ee6ea097442f9cbe5aaccc832e25; no voters:
I20260809 15:12:01.738986 21253 raft_consensus.cc:2781] T 05ac8774377744f99a97d9bd97368b5a P bbd73fc78a084ab5814444308af8da41 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:12:01.739419 21253 raft_consensus.cc:686] T 05ac8774377744f99a97d9bd97368b5a P bbd73fc78a084ab5814444308af8da41 [term 1 LEADER]: Becoming Leader. State: Replica: bbd73fc78a084ab5814444308af8da41, State: Running, Role: LEADER
I20260809 15:12:01.739926 21253 consensus_queue.cc:237] T 05ac8774377744f99a97d9bd97368b5a P bbd73fc78a084ab5814444308af8da41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11ee6ea097442f9cbe5aaccc832e25" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 34367 } } peers { permanent_uuid: "6b23c2a59c3a4078b5f970e0a70bfc64" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 34025 } } peers { permanent_uuid: "bbd73fc78a084ab5814444308af8da41" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 39665 } }
I20260809 15:12:01.744604 20415 catalog_manager.cc:5495] T 05ac8774377744f99a97d9bd97368b5a P bbd73fc78a084ab5814444308af8da41 reported cstate change: term changed from 0 to 1, leader changed from <none> to bbd73fc78a084ab5814444308af8da41 (127.24.183.195). New cstate: current_term: 1 leader_uuid: "bbd73fc78a084ab5814444308af8da41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11ee6ea097442f9cbe5aaccc832e25" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 34367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b23c2a59c3a4078b5f970e0a70bfc64" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 34025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbd73fc78a084ab5814444308af8da41" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 39665 } health_report { overall_health: HEALTHY } } }
I20260809 15:12:01.763482 21241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:01.758374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43816 (local address 127.24.183.195:39665)
0809 15:12:01.758535 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:01.758539 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:01.758556 (+ 17us) server_negotiation.cc:407] Connection header received
0809 15:12:01.758728 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:01.758732 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:01.758804 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:01.758940 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:12:01.759243 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:01.760056 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:01.761548 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:01.761942 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:01.762032 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:01.762531 (+ 499us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:12:01.763031 (+ 500us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:12:01.763137 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:01.763219 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 15:12:01.763284 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 15:12:01.771457 20660 raft_consensus.cc:1261] T 05ac8774377744f99a97d9bd97368b5a P 6b23c2a59c3a4078b5f970e0a70bfc64 [term 1 FOLLOWER]: Refusing update from remote peer bbd73fc78a084ab5814444308af8da41: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:12:01.771493 20903 raft_consensus.cc:1261] T 05ac8774377744f99a97d9bd97368b5a P fc11ee6ea097442f9cbe5aaccc832e25 [term 1 FOLLOWER]: Refusing update from remote peer bbd73fc78a084ab5814444308af8da41: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:12:01.772378 21257 consensus_queue.cc:1035] T 05ac8774377744f99a97d9bd97368b5a P bbd73fc78a084ab5814444308af8da41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc11ee6ea097442f9cbe5aaccc832e25" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 34367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:12:01.772776 21253 consensus_queue.cc:1035] T 05ac8774377744f99a97d9bd97368b5a P bbd73fc78a084ab5814444308af8da41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b23c2a59c3a4078b5f970e0a70bfc64" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 34025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 15:12:01.773765 21209 tablet.cc:2369] T 05ac8774377744f99a97d9bd97368b5a P bbd73fc78a084ab5814444308af8da41: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:12:01.786667 21261 mvcc.cc:205] Tried to move back new op lower bound from 7316636965966036992 to 7316636965850390528. Current Snapshot: MvccSnapshot[applied={T|T < 7316636965966036992}]
I20260809 15:12:01.796823 20415 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:12:01.797062 20415 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:12:01.800661 20415 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 10a5a952feb4482382cbfd43707ea59a: Sending DeleteTablet for 3 replicas of tablet 05ac8774377744f99a97d9bd97368b5a
I20260809 15:12:01.801873 20624 tablet_service.cc:1514] Processing DeleteTablet for tablet 05ac8774377744f99a97d9bd97368b5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:12:01 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:49004
I20260809 15:12:01.802287 21103 tablet_service.cc:1514] Processing DeleteTablet for tablet 05ac8774377744f99a97d9bd97368b5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:12:01 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:43808
I20260809 15:12:01.802600 20867 tablet_service.cc:1514] Processing DeleteTablet for tablet 05ac8774377744f99a97d9bd97368b5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:12:01 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:47324
I20260809 15:12:01.803407 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 20522
I20260809 15:12:01.813392 21275 tablet_replica.cc:330] T 05ac8774377744f99a97d9bd97368b5a P bbd73fc78a084ab5814444308af8da41: stopping tablet replica
I20260809 15:12:01.814028 21275 raft_consensus.cc:2229] T 05ac8774377744f99a97d9bd97368b5a P bbd73fc78a084ab5814444308af8da41 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:12:01.814886 21275 raft_consensus.cc:2258] T 05ac8774377744f99a97d9bd97368b5a P bbd73fc78a084ab5814444308af8da41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:12:01.815497 21273 tablet_replica.cc:330] T 05ac8774377744f99a97d9bd97368b5a P 6b23c2a59c3a4078b5f970e0a70bfc64: stopping tablet replica
I20260809 15:12:01.816447 21273 raft_consensus.cc:2229] T 05ac8774377744f99a97d9bd97368b5a P 6b23c2a59c3a4078b5f970e0a70bfc64 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:12:01.817437 21273 raft_consensus.cc:2258] T 05ac8774377744f99a97d9bd97368b5a P 6b23c2a59c3a4078b5f970e0a70bfc64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:12:01.818996 21275 ts_tablet_manager.cc:1905] T 05ac8774377744f99a97d9bd97368b5a P bbd73fc78a084ab5814444308af8da41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:12:01.822249 21276 tablet_replica.cc:330] T 05ac8774377744f99a97d9bd97368b5a P fc11ee6ea097442f9cbe5aaccc832e25: stopping tablet replica
I20260809 15:12:01.822682 21276 raft_consensus.cc:2229] T 05ac8774377744f99a97d9bd97368b5a P fc11ee6ea097442f9cbe5aaccc832e25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:12:01.823184 21276 raft_consensus.cc:2258] T 05ac8774377744f99a97d9bd97368b5a P fc11ee6ea097442f9cbe5aaccc832e25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:12:01.823915 21273 ts_tablet_manager.cc:1905] T 05ac8774377744f99a97d9bd97368b5a P 6b23c2a59c3a4078b5f970e0a70bfc64: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 15:12:01 dist-test-slave-nj21 krb5kdc[20341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.193@KRBTEST.COM
I20260809 15:12:01.830338 21276 ts_tablet_manager.cc:1905] T 05ac8774377744f99a97d9bd97368b5a P fc11ee6ea097442f9cbe5aaccc832e25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:12:01.833611 21275 ts_tablet_manager.cc:1918] T 05ac8774377744f99a97d9bd97368b5a P bbd73fc78a084ab5814444308af8da41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:12:01.837589 21273 ts_tablet_manager.cc:1918] T 05ac8774377744f99a97d9bd97368b5a P 6b23c2a59c3a4078b5f970e0a70bfc64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:12:01.837889 21275 log.cc:1196] T 05ac8774377744f99a97d9bd97368b5a P bbd73fc78a084ab5814444308af8da41: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-2/wal/wals/05ac8774377744f99a97d9bd97368b5a
I20260809 15:12:01.838510 21275 ts_tablet_manager.cc:1939] T 05ac8774377744f99a97d9bd97368b5a P bbd73fc78a084ab5814444308af8da41: Deleting consensus metadata
I20260809 15:12:01.838745 21273 log.cc:1196] T 05ac8774377744f99a97d9bd97368b5a P 6b23c2a59c3a4078b5f970e0a70bfc64: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-0/wal/wals/05ac8774377744f99a97d9bd97368b5a
I20260809 15:12:01.840833 20394 catalog_manager.cc:4841] TS bbd73fc78a084ab5814444308af8da41 (127.24.183.195:39665): tablet 05ac8774377744f99a97d9bd97368b5a (table test-table [id=be93960c17434759b9a0cfb620d6ad83]) successfully deleted
I20260809 15:12:01.841625 21273 ts_tablet_manager.cc:1939] T 05ac8774377744f99a97d9bd97368b5a P 6b23c2a59c3a4078b5f970e0a70bfc64: Deleting consensus metadata
I20260809 15:12:01.843016 21276 ts_tablet_manager.cc:1918] T 05ac8774377744f99a97d9bd97368b5a P fc11ee6ea097442f9cbe5aaccc832e25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:12:01.843183 21276 log.cc:1196] T 05ac8774377744f99a97d9bd97368b5a P fc11ee6ea097442f9cbe5aaccc832e25: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288021313853-25311-0/minicluster-data/ts-1/wal/wals/05ac8774377744f99a97d9bd97368b5a
I20260809 15:12:01.843366 20395 catalog_manager.cc:4841] TS 6b23c2a59c3a4078b5f970e0a70bfc64 (127.24.183.193:34025): tablet 05ac8774377744f99a97d9bd97368b5a (table test-table [id=be93960c17434759b9a0cfb620d6ad83]) successfully deleted
I20260809 15:12:01.843712 21276 ts_tablet_manager.cc:1939] T 05ac8774377744f99a97d9bd97368b5a P fc11ee6ea097442f9cbe5aaccc832e25: Deleting consensus metadata
I20260809 15:12:01.845316 20391 catalog_manager.cc:4841] TS fc11ee6ea097442f9cbe5aaccc832e25 (127.24.183.194:34367): tablet 05ac8774377744f99a97d9bd97368b5a (table test-table [id=be93960c17434759b9a0cfb620d6ad83]) successfully deleted
I20260809 15:12:01.846290 21238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:01.805056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49008 (local address 127.24.183.193:34025)
0809 15:12:01.805258 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:01.805265 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:01.815282 (+ 10017us) server_negotiation.cc:407] Connection header received
0809 15:12:01.815771 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:01.815776 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:01.815862 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:01.815937 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:01.818359 (+ 2422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:01.819472 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:01.823569 (+ 4097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:01.824210 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:01.824319 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:01.832927 (+ 8608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:01.832942 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:01.832960 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:12:01.833011 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:01.836925 (+ 3914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:01.843856 (+ 6931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:01.843866 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:01.843869 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:01.844038 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:01.844526 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:01.844532 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:01.844535 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:01.845015 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:01.845313 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:01.845695 (+ 382us) server_negotiation.cc:299] Negotiation successful
0809 15:12:01.845930 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 15:12:01.846994 20729 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:12:01.936312 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 20522
I20260809 15:12:01.961122 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 20748
Aug 09 15:12:01 dist-test-slave-nj21 krb5kdc[20341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.194@KRBTEST.COM
I20260809 15:12:01.976990 21235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:01.961946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47326 (local address 127.24.183.194:34367)
0809 15:12:01.962133 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:01.962139 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:01.962215 (+ 76us) server_negotiation.cc:407] Connection header received
0809 15:12:01.962435 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:01.962439 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:01.962519 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:01.962607 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:01.964058 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:01.964797 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:01.966134 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:01.966559 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:01.966690 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:01.971293 (+ 4603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:01.971309 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:01.971326 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:12:01.971362 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:01.974232 (+ 2870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:01.974870 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:01.974881 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:01.974884 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:01.975014 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:01.975376 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:01.975381 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:01.975385 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:01.975832 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:01.976134 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:01.976425 (+ 291us) server_negotiation.cc:299] Negotiation successful
0809 15:12:01.976638 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 15:12:01.977625 20963 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:12:02.066955 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 20748
I20260809 15:12:02.092427 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 20986
Aug 09 15:12:02 dist-test-slave-nj21 krb5kdc[20341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.195@KRBTEST.COM
I20260809 15:12:02.109695 21241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:02.093207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43824 (local address 127.24.183.195:39665)
0809 15:12:02.093759 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:02.093765 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:02.093785 (+ 20us) server_negotiation.cc:407] Connection header received
0809 15:12:02.093853 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:02.093863 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:02.093962 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:02.094077 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:02.095380 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:02.096661 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:02.097892 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:02.098576 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:02.098706 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:02.103404 (+ 4698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:02.103420 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:02.103438 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:12:02.103482 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:02.106565 (+ 3083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:02.107470 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:02.107478 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:02.107481 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:02.107656 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:02.108104 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:02.108107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:02.108109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:02.108516 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:02.108787 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:02.109140 (+ 353us) server_negotiation.cc:299] Negotiation successful
0809 15:12:02.109307 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406}
I20260809 15:12:02.110627 21202 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:12:02.237800 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 20986
I20260809 15:12:02.263047 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 20359
I20260809 15:12:02.264238 20469 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:12:02.376369 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 20359
2026-08-09T15:12:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4516 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T15:12:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:12:02Z Disabled control of system clock
I20260809 15:12:02.467936 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:38793
--webserver_interface=127.24.183.254
--webserver_port=0
--builtin_ntp_servers=127.24.183.212:37671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:38793
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 15:12:02.801180 21287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:02.801463 21287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:02.801560 21287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:02.808903 21287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:12:02.809010 21287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:02.809059 21287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:02.809124 21287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:12:02.809175 21287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:12:02.818869 21287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:37671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:38793
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:38793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:02.820153 21287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:02.821986 21287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:02.833618 21299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:02.833725 21295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:02.833716 21302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:02.834576 21287 server_base.cc:1034] running on GCE node
I20260809 15:12:02.835383 21287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:02.836809 21287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:02.838012 21287 hybrid_clock.cc:648] HybridClock initialized: now 1786288322837963 us; error 36 us; skew 500 ppm
I20260809 15:12:02.838352 21287 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:12:02.840577 21287 webserver.cc:460] Webserver started at http://127.24.183.254:38885/ using document root <none> and password file <none>
I20260809 15:12:02.841243 21287 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:02.841346 21287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:02.841655 21287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:02.844606 21287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "a1b3c3071ad3496e9b6212d9b7aa9d72"
format_stamp: "Formatted at 2026-08-09 15:12:02 on dist-test-slave-nj21"
server_key: "4cb75fdfbd067c7c6c17e1477ebcf5d1"
server_key_iv: "50ef350afd96e152fe2ff7bb80b3af2c"
server_key_version: "encryptionkey@0"
I20260809 15:12:02.845424 21287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "a1b3c3071ad3496e9b6212d9b7aa9d72"
format_stamp: "Formatted at 2026-08-09 15:12:02 on dist-test-slave-nj21"
server_key: "4cb75fdfbd067c7c6c17e1477ebcf5d1"
server_key_iv: "50ef350afd96e152fe2ff7bb80b3af2c"
server_key_version: "encryptionkey@0"
I20260809 15:12:02.850455 21287 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260809 15:12:02.854084 21314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:02.855360 21287 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:12:02.855525 21287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "a1b3c3071ad3496e9b6212d9b7aa9d72"
format_stamp: "Formatted at 2026-08-09 15:12:02 on dist-test-slave-nj21"
server_key: "4cb75fdfbd067c7c6c17e1477ebcf5d1"
server_key_iv: "50ef350afd96e152fe2ff7bb80b3af2c"
server_key_version: "encryptionkey@0"
I20260809 15:12:02.855768 21287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:02.873193 21287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:02.873965 21287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:02.874215 21287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:02.891846 21287 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:38793
I20260809 15:12:02.891867 21400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:38793 every 8 connection(s)
I20260809 15:12:02.893188 21287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:12:02.896905 21402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:02.902099 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 21287
I20260809 15:12:02.902271 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:12:02.902825 25311 external_mini_cluster.cc:1428] Setting key 669d75f5972c5656463dcb6d5496dffb
I20260809 15:12:02.908185 21402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1b3c3071ad3496e9b6212d9b7aa9d72: Bootstrap starting.
I20260809 15:12:02.911720 21402 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1b3c3071ad3496e9b6212d9b7aa9d72: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:02.912822 21402 log.cc:824] T 00000000000000000000000000000000 P a1b3c3071ad3496e9b6212d9b7aa9d72: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:02.915920 21402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1b3c3071ad3496e9b6212d9b7aa9d72: No bootstrap required, opened a new log
I20260809 15:12:02.916695 21407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:02.904754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46104 (local address 127.24.183.254:38793)
0809 15:12:02.905727 (+ 973us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:02.905738 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:02.905786 (+ 48us) server_negotiation.cc:407] Connection header received
0809 15:12:02.907519 (+ 1733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:02.907542 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:02.908018 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:02.908373 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:02.908907 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:02.910243 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:02.912143 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:02.912822 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:02.912978 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:02.914261 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:02.914274 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:02.914291 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:02.914853 (+ 562us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:02.914937 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:02.914961 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:02.915064 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:02.915298 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 15:12:02.915628 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":365,"threads_started":1}
I20260809 15:12:02.923092 21402 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1b3c3071ad3496e9b6212d9b7aa9d72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1b3c3071ad3496e9b6212d9b7aa9d72" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 38793 } }
I20260809 15:12:02.923432 21402 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1b3c3071ad3496e9b6212d9b7aa9d72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:02.923542 21402 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1b3c3071ad3496e9b6212d9b7aa9d72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1b3c3071ad3496e9b6212d9b7aa9d72, State: Initialized, Role: FOLLOWER
I20260809 15:12:02.924078 21402 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1b3c3071ad3496e9b6212d9b7aa9d72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1b3c3071ad3496e9b6212d9b7aa9d72" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 38793 } }
I20260809 15:12:02.924294 21402 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1b3c3071ad3496e9b6212d9b7aa9d72 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:12:02.924432 21402 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1b3c3071ad3496e9b6212d9b7aa9d72 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:12:02.924589 21402 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1b3c3071ad3496e9b6212d9b7aa9d72 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:02.926671 21402 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1b3c3071ad3496e9b6212d9b7aa9d72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1b3c3071ad3496e9b6212d9b7aa9d72" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 38793 } }
I20260809 15:12:02.927126 21402 leader_election.cc:302] T 00000000000000000000000000000000 P a1b3c3071ad3496e9b6212d9b7aa9d72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1b3c3071ad3496e9b6212d9b7aa9d72; no voters:
I20260809 15:12:02.927647 21402 leader_election.cc:288] T 00000000000000000000000000000000 P a1b3c3071ad3496e9b6212d9b7aa9d72 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:12:02.927843 21410 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1b3c3071ad3496e9b6212d9b7aa9d72 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:12:02.928535 21410 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1b3c3071ad3496e9b6212d9b7aa9d72 [term 1 LEADER]: Becoming Leader. State: Replica: a1b3c3071ad3496e9b6212d9b7aa9d72, State: Running, Role: LEADER
I20260809 15:12:02.929067 21410 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1b3c3071ad3496e9b6212d9b7aa9d72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1b3c3071ad3496e9b6212d9b7aa9d72" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 38793 } }
I20260809 15:12:02.929641 21402 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1b3c3071ad3496e9b6212d9b7aa9d72 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:12:02.931582 21411 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1b3c3071ad3496e9b6212d9b7aa9d72 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1b3c3071ad3496e9b6212d9b7aa9d72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1b3c3071ad3496e9b6212d9b7aa9d72" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 38793 } } }
I20260809 15:12:02.931931 21411 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1b3c3071ad3496e9b6212d9b7aa9d72 [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:02.933838 21417 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:12:02.933717 21412 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1b3c3071ad3496e9b6212d9b7aa9d72 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1b3c3071ad3496e9b6212d9b7aa9d72. Latest consensus state: current_term: 1 leader_uuid: "a1b3c3071ad3496e9b6212d9b7aa9d72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1b3c3071ad3496e9b6212d9b7aa9d72" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 38793 } } }
I20260809 15:12:02.934017 21412 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1b3c3071ad3496e9b6212d9b7aa9d72 [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:02.938247 21417 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:12:02.950126 21417 catalog_manager.cc:1347] Generated new cluster ID: 35e2ed948cc14111a7fc7344292867a7
I20260809 15:12:02.950256 21417 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:12:02.988402 21417 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:12:02.989970 21417 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:12:03.002907 21417 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1b3c3071ad3496e9b6212d9b7aa9d72: Generated new TSK 0
I20260809 15:12:03.003877 21417 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:12:03.022645 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.193:0
--local_ip_for_outbound_sockets=127.24.183.193
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:38793
--builtin_ntp_servers=127.24.183.212:37671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 15:12:03.368968 21442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:03.369236 21442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:03.369366 21442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:03.376539 21442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:03.376626 21442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:03.376771 21442 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.193
I20260809 15:12:03.385812 21442 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:37671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:38793
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.193
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:03.387108 21442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:03.389025 21442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:03.401911 21455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:03.401980 21453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:03.401955 21452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:03.403079 21442 server_base.cc:1034] running on GCE node
I20260809 15:12:03.403754 21442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:03.404707 21442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:03.405916 21442 hybrid_clock.cc:648] HybridClock initialized: now 1786288323405870 us; error 33 us; skew 500 ppm
I20260809 15:12:03.406224 21442 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:12:03.408465 21442 webserver.cc:460] Webserver started at http://127.24.183.193:37741/ using document root <none> and password file <none>
I20260809 15:12:03.409117 21442 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:03.409240 21442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:03.409566 21442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:03.412551 21442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-0/data/instance:
uuid: "a42c0ab1eca8460bb35b3717efdbc76a"
format_stamp: "Formatted at 2026-08-09 15:12:03 on dist-test-slave-nj21"
server_key: "2b1c4b124cbbd4509f3b00e9cffc7d72"
server_key_iv: "0399faaab0133c8e685524d66c8cfe50"
server_key_version: "encryptionkey@0"
I20260809 15:12:03.413350 21442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance:
uuid: "a42c0ab1eca8460bb35b3717efdbc76a"
format_stamp: "Formatted at 2026-08-09 15:12:03 on dist-test-slave-nj21"
server_key: "2b1c4b124cbbd4509f3b00e9cffc7d72"
server_key_iv: "0399faaab0133c8e685524d66c8cfe50"
server_key_version: "encryptionkey@0"
I20260809 15:12:03.418646 21442 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 15:12:03.422209 21464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:03.423465 21442 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:12:03.423614 21442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-0/wal
uuid: "a42c0ab1eca8460bb35b3717efdbc76a"
format_stamp: "Formatted at 2026-08-09 15:12:03 on dist-test-slave-nj21"
server_key: "2b1c4b124cbbd4509f3b00e9cffc7d72"
server_key_iv: "0399faaab0133c8e685524d66c8cfe50"
server_key_version: "encryptionkey@0"
I20260809 15:12:03.423849 21442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:03.447029 21442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:03.447983 21442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:03.448293 21442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:03.449312 21442 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:12:03.450855 21442 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:12:03.450947 21442 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:03.451056 21442 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:12:03.451131 21442 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:03.481455 21442 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.193:43433
I20260809 15:12:03.481520 21640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.193:43433 every 8 connection(s)
I20260809 15:12:03.482682 21442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
I20260809 15:12:03.483402 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 21442
I20260809 15:12:03.483587 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance
I20260809 15:12:03.484172 25311 external_mini_cluster.cc:1428] Setting key 013661386691fe7ab5112ac3e5d65758
I20260809 15:12:03.487167 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.194:0
--local_ip_for_outbound_sockets=127.24.183.194
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:38793
--builtin_ntp_servers=127.24.183.212:37671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 15:12:03.495374 21644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:03.485408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:36403 (local address 127.24.183.254:38793)
0809 15:12:03.485876 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:03.485883 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:03.487352 (+ 1469us) server_negotiation.cc:407] Connection header received
0809 15:12:03.489170 (+ 1818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:03.489176 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:03.489258 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:03.489353 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:03.490426 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:03.491167 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:03.492745 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:03.493157 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:03.493270 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:03.494533 (+ 1263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:03.494539 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:03.494542 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:03.494561 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:03.494604 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:03.494616 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:03.494691 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:03.494903 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 15:12:03.495006 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":222,"threads_started":1}
I20260809 15:12:03.496377 21646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:03.486129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:38793 (local address 127.24.183.193:36403)
0809 15:12:03.487158 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:03.487225 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:03.488916 (+ 1691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:03.489608 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:03.489617 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:03.489709 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:12:03.490264 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:03.490282 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:03.491326 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:03.491330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:03.492597 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:03.492611 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:03.493379 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:03.493387 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:03.493602 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:03.494239 (+ 637us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:12:03.494242 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:12:03.494280 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:12:03.494359 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:12:03.494704 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:12:03.494716 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:12:03.494725 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 15:12:03.494876 (+ 151us) client_negotiation.cc:240] Negotiation successful
0809 15:12:03.495264 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":328,"threads_started":1}
I20260809 15:12:03.497797 21641 heartbeater.cc:347] Connected to a master server at 127.24.183.254:38793
I20260809 15:12:03.498109 21641 heartbeater.cc:464] Registering TS with master...
I20260809 15:12:03.498967 21641 heartbeater.cc:511] Master 127.24.183.254:38793 requested a full tablet report, sending...
I20260809 15:12:03.501063 21344 ts_manager.cc:194] Registered new tserver with Master: a42c0ab1eca8460bb35b3717efdbc76a (127.24.183.193:43433)
I20260809 15:12:03.502539 21344 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.183.193:36403
W20260809 15:12:03.830013 21647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:03.830443 21647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:03.830596 21647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:03.838204 21647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:03.838312 21647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:03.838456 21647 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.194
I20260809 15:12:03.848146 21647 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:37671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:38793
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.194
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:03.849586 21647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:03.851421 21647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:03.864456 21662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:03.864436 21665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:03.864650 21661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:03.865307 21647 server_base.cc:1034] running on GCE node
I20260809 15:12:03.865917 21647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:03.866861 21647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:03.868068 21647 hybrid_clock.cc:648] HybridClock initialized: now 1786288323868003 us; error 52 us; skew 500 ppm
I20260809 15:12:03.868377 21647 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:12:03.870541 21647 webserver.cc:460] Webserver started at http://127.24.183.194:43013/ using document root <none> and password file <none>
I20260809 15:12:03.871230 21647 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:03.871335 21647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:03.871651 21647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:03.874583 21647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-1/data/instance:
uuid: "43c3ef5a5b8c478698f66060542192fd"
format_stamp: "Formatted at 2026-08-09 15:12:03 on dist-test-slave-nj21"
server_key: "238e351799828e371e76860689965e09"
server_key_iv: "a3e07da38a803ed8984f4289e5cbf115"
server_key_version: "encryptionkey@0"
I20260809 15:12:03.875355 21647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance:
uuid: "43c3ef5a5b8c478698f66060542192fd"
format_stamp: "Formatted at 2026-08-09 15:12:03 on dist-test-slave-nj21"
server_key: "238e351799828e371e76860689965e09"
server_key_iv: "a3e07da38a803ed8984f4289e5cbf115"
server_key_version: "encryptionkey@0"
I20260809 15:12:03.880733 21647 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260809 15:12:03.884243 21674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:03.885479 21647 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:12:03.885636 21647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-1/wal
uuid: "43c3ef5a5b8c478698f66060542192fd"
format_stamp: "Formatted at 2026-08-09 15:12:03 on dist-test-slave-nj21"
server_key: "238e351799828e371e76860689965e09"
server_key_iv: "a3e07da38a803ed8984f4289e5cbf115"
server_key_version: "encryptionkey@0"
I20260809 15:12:03.885838 21647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:03.912822 21647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:03.913669 21647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:03.913975 21647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:03.914954 21647 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:12:03.916471 21647 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:12:03.916570 21647 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:03.916682 21647 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:12:03.916752 21647 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:03.947263 21647 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.194:42555
I20260809 15:12:03.947316 21871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.194:42555 every 8 connection(s)
I20260809 15:12:03.948578 21647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
I20260809 15:12:03.958531 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 21647
I20260809 15:12:03.958698 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance
I20260809 15:12:03.959137 25311 external_mini_cluster.cc:1428] Setting key 09a41f3db3a8a41d345cac2ca3bc7423
I20260809 15:12:03.960906 21644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:03.951076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:47429 (local address 127.24.183.254:38793)
0809 15:12:03.951258 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:03.951263 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:03.952725 (+ 1462us) server_negotiation.cc:407] Connection header received
0809 15:12:03.954509 (+ 1784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:03.954515 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:03.954590 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:03.954678 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:03.955727 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:03.956464 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:03.957963 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:03.958369 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:03.958480 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:03.959990 (+ 1510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:03.960017 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:03.960024 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:03.960061 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:03.960104 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:03.960132 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:03.960235 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:03.960461 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 15:12:03.960565 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 15:12:03.961953 21874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:03.951647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:38793 (local address 127.24.183.194:47429)
0809 15:12:03.952581 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:03.952617 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:03.954239 (+ 1622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:03.954906 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:03.954913 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:03.954998 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:12:03.955543 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:03.955560 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:03.956613 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:03.956617 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:03.957818 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:03.957830 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:03.958582 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:03.958589 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:03.958850 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:03.959626 (+ 776us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:12:03.959629 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:12:03.959710 (+ 81us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:12:03.959801 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:12:03.960249 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:12:03.960261 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:12:03.960270 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 15:12:03.960450 (+ 180us) client_negotiation.cc:240] Negotiation successful
0809 15:12:03.960830 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":279,"threads_started":1}
I20260809 15:12:03.962098 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.195:0
--local_ip_for_outbound_sockets=127.24.183.195
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:38793
--builtin_ntp_servers=127.24.183.212:37671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 15:12:03.963416 21872 heartbeater.cc:347] Connected to a master server at 127.24.183.254:38793
I20260809 15:12:03.963711 21872 heartbeater.cc:464] Registering TS with master...
I20260809 15:12:03.964447 21872 heartbeater.cc:511] Master 127.24.183.254:38793 requested a full tablet report, sending...
I20260809 15:12:03.965819 21344 ts_manager.cc:194] Registered new tserver with Master: 43c3ef5a5b8c478698f66060542192fd (127.24.183.194:42555)
I20260809 15:12:03.966768 21344 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.183.194:47429
W20260809 15:12:04.304111 21875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:04.304373 21875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:04.304486 21875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:04.311483 21875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:04.311574 21875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:04.311740 21875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.195
I20260809 15:12:04.320518 21875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:37671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:38793
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.195
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:04.321801 21875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:04.323580 21875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:04.337047 21892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:04.337301 21888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:04.337069 21886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:04.338140 21875 server_base.cc:1034] running on GCE node
I20260809 15:12:04.338778 21875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:04.339845 21875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:04.341082 21875 hybrid_clock.cc:648] HybridClock initialized: now 1786288324340999 us; error 61 us; skew 500 ppm
I20260809 15:12:04.341487 21875 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:12:04.343847 21875 webserver.cc:460] Webserver started at http://127.24.183.195:45983/ using document root <none> and password file <none>
I20260809 15:12:04.344533 21875 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:04.344638 21875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:04.344944 21875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:04.347935 21875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-2/data/instance:
uuid: "e253c5347150417495a6b6b50d19524b"
format_stamp: "Formatted at 2026-08-09 15:12:04 on dist-test-slave-nj21"
server_key: "a9f02b62962657ddb3cf6a8c647dba46"
server_key_iv: "5da870474c989207facc3cdc39736910"
server_key_version: "encryptionkey@0"
I20260809 15:12:04.348706 21875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance:
uuid: "e253c5347150417495a6b6b50d19524b"
format_stamp: "Formatted at 2026-08-09 15:12:04 on dist-test-slave-nj21"
server_key: "a9f02b62962657ddb3cf6a8c647dba46"
server_key_iv: "5da870474c989207facc3cdc39736910"
server_key_version: "encryptionkey@0"
I20260809 15:12:04.353817 21875 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 15:12:04.357506 21901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:04.358774 21875 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 15:12:04.358930 21875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-2/wal
uuid: "e253c5347150417495a6b6b50d19524b"
format_stamp: "Formatted at 2026-08-09 15:12:04 on dist-test-slave-nj21"
server_key: "a9f02b62962657ddb3cf6a8c647dba46"
server_key_iv: "5da870474c989207facc3cdc39736910"
server_key_version: "encryptionkey@0"
I20260809 15:12:04.359138 21875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:04.385035 21875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:04.385821 21875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:04.386101 21875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:04.387075 21875 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:12:04.388515 21875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:12:04.388610 21875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:04.388741 21875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:12:04.388813 21875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:04.420331 21875 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.195:41387
I20260809 15:12:04.420414 22087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.195:41387 every 8 connection(s)
I20260809 15:12:04.421602 21875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
I20260809 15:12:04.426319 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 21875
I20260809 15:12:04.426528 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance
I20260809 15:12:04.427157 25311 external_mini_cluster.cc:1428] Setting key 83da0148bc0c7df799e540a64e57906c
I20260809 15:12:04.434669 21644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:04.424319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:54613 (local address 127.24.183.254:38793)
0809 15:12:04.424536 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:04.424543 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:04.425927 (+ 1384us) server_negotiation.cc:407] Connection header received
0809 15:12:04.428386 (+ 2459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:04.428404 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:04.428488 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:04.428610 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:04.429798 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:04.430662 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:04.432289 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:04.432692 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:04.432784 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:04.434027 (+ 1243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:04.434032 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:04.434035 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:04.434054 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:04.434085 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:04.434095 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:04.434159 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:04.434334 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 15:12:04.434409 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 15:12:04.435716 22092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:04.424814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:38793 (local address 127.24.183.195:54613)
0809 15:12:04.425750 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:04.425790 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:04.428131 (+ 2341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:04.428836 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:04.428848 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:04.428961 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:12:04.429620 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:04.429642 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:04.430806 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:04.430811 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:04.432162 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:04.432176 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:04.432873 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:04.432879 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:04.433081 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:04.433757 (+ 676us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:12:04.433759 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:12:04.433805 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:12:04.433882 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:12:04.434168 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:12:04.434174 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:12:04.434180 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 15:12:04.434331 (+ 151us) client_negotiation.cc:240] Negotiation successful
0809 15:12:04.434600 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":287,"threads_started":1}
I20260809 15:12:04.437110 22089 heartbeater.cc:347] Connected to a master server at 127.24.183.254:38793
I20260809 15:12:04.437412 22089 heartbeater.cc:464] Registering TS with master...
I20260809 15:12:04.438165 22089 heartbeater.cc:511] Master 127.24.183.254:38793 requested a full tablet report, sending...
I20260809 15:12:04.439571 21344 ts_manager.cc:194] Registered new tserver with Master: e253c5347150417495a6b6b50d19524b (127.24.183.195:41387)
I20260809 15:12:04.440554 21344 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.183.195:54613
I20260809 15:12:04.442452 25311 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:12:04.451826 21644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:04.446565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46118 (local address 127.24.183.254:38793)
0809 15:12:04.446732 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:04.446737 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:04.447162 (+ 425us) server_negotiation.cc:407] Connection header received
0809 15:12:04.447286 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:04.447289 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:04.447361 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:04.447442 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:04.447937 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:04.448614 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:04.449924 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:04.450356 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:04.450446 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:04.451289 (+ 843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:04.451294 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:04.451297 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:04.451314 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:04.451344 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:04.451355 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:04.451419 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:04.451492 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 15:12:04.451560 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 15:12:04.455271 21342 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46118:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:12:04.462219 21342 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:12:04.504504 22107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:04.473659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:42555 (local address 127.0.0.1:49496)
0809 15:12:04.496348 (+ 22689us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:04.496401 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:04.496580 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:04.497764 (+ 1184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:04.497771 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:04.497800 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:12:04.498210 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:04.498224 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:04.500039 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:04.500044 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:04.502025 (+ 1981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:04.502039 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:04.503311 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:04.503320 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:04.503695 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:04.503750 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 15:12:04.503861 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 15:12:04.504070 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22448,"mutex_wait_us":111,"thread_start_us":24068,"threads_started":1}
I20260809 15:12:04.504438 22109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:04.474259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49496 (local address 127.24.183.194:42555)
0809 15:12:04.495598 (+ 21339us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:04.495608 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:04.496792 (+ 1184us) server_negotiation.cc:407] Connection header received
0809 15:12:04.496953 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:04.496961 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:04.497209 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:04.497388 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:12:04.498446 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:04.499923 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:04.502199 (+ 2276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:04.503111 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:04.503251 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:04.503478 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:04.503932 (+ 454us) server_negotiation.cc:299] Negotiation successful
0809 15:12:04.504098 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21142,"thread_start_us":21352,"threads_started":1}
I20260809 15:12:04.505997 21641 heartbeater.cc:503] Master 127.24.183.254:38793 was elected leader, sending a full tablet report...
I20260809 15:12:04.507475 22105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:04.473735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:41387 (local address 127.0.0.1:38026)
0809 15:12:04.497365 (+ 23630us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:04.497381 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:04.497486 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:04.499198 (+ 1712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:04.499203 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:04.499228 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:12:04.499631 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:04.499642 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:04.501797 (+ 2155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:04.501803 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:04.504101 (+ 2298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:04.504116 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:04.506734 (+ 2618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:04.506741 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:04.506903 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:04.506936 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 15:12:04.507013 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 15:12:04.507141 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23486,"mutex_wait_us":51,"thread_start_us":25340,"threads_started":1}
I20260809 15:12:04.508028 22110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:04.474605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36090 (local address 127.24.183.193:43433)
0809 15:12:04.496955 (+ 22350us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:04.496963 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:04.499475 (+ 2512us) server_negotiation.cc:407] Connection header received
0809 15:12:04.499657 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:04.499663 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:04.499903 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:04.500083 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:12:04.501068 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:04.502539 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:04.505172 (+ 2633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:04.506558 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:04.506726 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:04.506916 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:04.507478 (+ 562us) server_negotiation.cc:299] Negotiation successful
0809 15:12:04.507644 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22163,"thread_start_us":22947,"threads_started":1}
I20260809 15:12:04.508421 22108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:04.474028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38026 (local address 127.24.183.195:41387)
0809 15:12:04.498620 (+ 24592us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:04.498637 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:04.498670 (+ 33us) server_negotiation.cc:407] Connection header received
0809 15:12:04.498765 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:04.498772 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:04.498982 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:04.499781 (+ 799us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:12:04.500088 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:04.501641 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:04.504340 (+ 2699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:04.505663 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:04.506400 (+ 737us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:04.506611 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:04.507911 (+ 1300us) server_negotiation.cc:299] Negotiation successful
0809 15:12:04.508068 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24424,"thread_start_us":24273,"threads_started":1}
I20260809 15:12:04.508606 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:04.474049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:43433 (local address 127.0.0.1:36090)
0809 15:12:04.499357 (+ 25308us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:04.499372 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:04.499503 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:04.500327 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:04.500332 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:04.500357 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:12:04.500831 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:04.500844 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:04.502736 (+ 1892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:04.502741 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:04.504949 (+ 2208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:04.504965 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:04.507096 (+ 2131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:04.507103 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:04.507272 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:04.507303 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 15:12:04.508254 (+ 951us) client_negotiation.cc:240] Negotiation successful
0809 15:12:04.508355 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25191,"mutex_wait_us":1,"thread_start_us":21930,"threads_started":1}
I20260809 15:12:04.509339 21764 tablet_service.cc:1467] Processing CreateTablet for tablet 4a89276adf354864a22670c0805a2a9f (DEFAULT_TABLE table=test-table [id=5284316b7c264ba085a7848030deb945]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:12:04.510634 21764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a89276adf354864a22670c0805a2a9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:04.512082 21981 tablet_service.cc:1467] Processing CreateTablet for tablet 4a89276adf354864a22670c0805a2a9f (DEFAULT_TABLE table=test-table [id=5284316b7c264ba085a7848030deb945]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:12:04.512081 21537 tablet_service.cc:1467] Processing CreateTablet for tablet 4a89276adf354864a22670c0805a2a9f (DEFAULT_TABLE table=test-table [id=5284316b7c264ba085a7848030deb945]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:12:04.513463 21537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a89276adf354864a22670c0805a2a9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:04.513464 21981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a89276adf354864a22670c0805a2a9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:04.521775 22111 tablet_bootstrap.cc:492] T 4a89276adf354864a22670c0805a2a9f P 43c3ef5a5b8c478698f66060542192fd: Bootstrap starting.
I20260809 15:12:04.524791 22111 tablet_bootstrap.cc:654] T 4a89276adf354864a22670c0805a2a9f P 43c3ef5a5b8c478698f66060542192fd: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:04.525758 22111 log.cc:824] T 4a89276adf354864a22670c0805a2a9f P 43c3ef5a5b8c478698f66060542192fd: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:04.528232 22115 tablet_bootstrap.cc:492] T 4a89276adf354864a22670c0805a2a9f P e253c5347150417495a6b6b50d19524b: Bootstrap starting.
I20260809 15:12:04.531792 22113 tablet_bootstrap.cc:492] T 4a89276adf354864a22670c0805a2a9f P a42c0ab1eca8460bb35b3717efdbc76a: Bootstrap starting.
I20260809 15:12:04.531908 22111 tablet_bootstrap.cc:492] T 4a89276adf354864a22670c0805a2a9f P 43c3ef5a5b8c478698f66060542192fd: No bootstrap required, opened a new log
I20260809 15:12:04.531982 22115 tablet_bootstrap.cc:654] T 4a89276adf354864a22670c0805a2a9f P e253c5347150417495a6b6b50d19524b: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:04.532182 22111 ts_tablet_manager.cc:1397] T 4a89276adf354864a22670c0805a2a9f P 43c3ef5a5b8c478698f66060542192fd: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260809 15:12:04.533198 22115 log.cc:824] T 4a89276adf354864a22670c0805a2a9f P e253c5347150417495a6b6b50d19524b: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:04.534746 22113 tablet_bootstrap.cc:654] T 4a89276adf354864a22670c0805a2a9f P a42c0ab1eca8460bb35b3717efdbc76a: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:04.535851 22113 log.cc:824] T 4a89276adf354864a22670c0805a2a9f P a42c0ab1eca8460bb35b3717efdbc76a: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:04.536867 22115 tablet_bootstrap.cc:492] T 4a89276adf354864a22670c0805a2a9f P e253c5347150417495a6b6b50d19524b: No bootstrap required, opened a new log
I20260809 15:12:04.537115 22115 ts_tablet_manager.cc:1397] T 4a89276adf354864a22670c0805a2a9f P e253c5347150417495a6b6b50d19524b: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260809 15:12:04.538247 22113 tablet_bootstrap.cc:492] T 4a89276adf354864a22670c0805a2a9f P a42c0ab1eca8460bb35b3717efdbc76a: No bootstrap required, opened a new log
I20260809 15:12:04.538477 22113 ts_tablet_manager.cc:1397] T 4a89276adf354864a22670c0805a2a9f P a42c0ab1eca8460bb35b3717efdbc76a: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.005s
I20260809 15:12:04.542268 22111 raft_consensus.cc:348] T 4a89276adf354864a22670c0805a2a9f P 43c3ef5a5b8c478698f66060542192fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e253c5347150417495a6b6b50d19524b" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 41387 } } peers { permanent_uuid: "43c3ef5a5b8c478698f66060542192fd" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 42555 } } peers { permanent_uuid: "a42c0ab1eca8460bb35b3717efdbc76a" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 43433 } }
I20260809 15:12:04.542703 22111 raft_consensus.cc:374] T 4a89276adf354864a22670c0805a2a9f P 43c3ef5a5b8c478698f66060542192fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:04.542799 22111 raft_consensus.cc:729] T 4a89276adf354864a22670c0805a2a9f P 43c3ef5a5b8c478698f66060542192fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43c3ef5a5b8c478698f66060542192fd, State: Initialized, Role: FOLLOWER
I20260809 15:12:04.543414 22111 consensus_queue.cc:260] T 4a89276adf354864a22670c0805a2a9f P 43c3ef5a5b8c478698f66060542192fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e253c5347150417495a6b6b50d19524b" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 41387 } } peers { permanent_uuid: "43c3ef5a5b8c478698f66060542192fd" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 42555 } } peers { permanent_uuid: "a42c0ab1eca8460bb35b3717efdbc76a" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 43433 } }
I20260809 15:12:04.544648 21872 heartbeater.cc:503] Master 127.24.183.254:38793 was elected leader, sending a full tablet report...
I20260809 15:12:04.545246 22111 ts_tablet_manager.cc:1428] T 4a89276adf354864a22670c0805a2a9f P 43c3ef5a5b8c478698f66060542192fd: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
I20260809 15:12:04.546108 22113 raft_consensus.cc:348] T 4a89276adf354864a22670c0805a2a9f P a42c0ab1eca8460bb35b3717efdbc76a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e253c5347150417495a6b6b50d19524b" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 41387 } } peers { permanent_uuid: "43c3ef5a5b8c478698f66060542192fd" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 42555 } } peers { permanent_uuid: "a42c0ab1eca8460bb35b3717efdbc76a" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 43433 } }
I20260809 15:12:04.546569 22113 raft_consensus.cc:374] T 4a89276adf354864a22670c0805a2a9f P a42c0ab1eca8460bb35b3717efdbc76a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:04.546722 22113 raft_consensus.cc:729] T 4a89276adf354864a22670c0805a2a9f P a42c0ab1eca8460bb35b3717efdbc76a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a42c0ab1eca8460bb35b3717efdbc76a, State: Initialized, Role: FOLLOWER
I20260809 15:12:04.547366 22113 consensus_queue.cc:260] T 4a89276adf354864a22670c0805a2a9f P a42c0ab1eca8460bb35b3717efdbc76a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e253c5347150417495a6b6b50d19524b" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 41387 } } peers { permanent_uuid: "43c3ef5a5b8c478698f66060542192fd" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 42555 } } peers { permanent_uuid: "a42c0ab1eca8460bb35b3717efdbc76a" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 43433 } }
I20260809 15:12:04.547856 22115 raft_consensus.cc:348] T 4a89276adf354864a22670c0805a2a9f P e253c5347150417495a6b6b50d19524b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e253c5347150417495a6b6b50d19524b" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 41387 } } peers { permanent_uuid: "43c3ef5a5b8c478698f66060542192fd" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 42555 } } peers { permanent_uuid: "a42c0ab1eca8460bb35b3717efdbc76a" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 43433 } }
I20260809 15:12:04.548302 22115 raft_consensus.cc:374] T 4a89276adf354864a22670c0805a2a9f P e253c5347150417495a6b6b50d19524b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:04.548421 22115 raft_consensus.cc:729] T 4a89276adf354864a22670c0805a2a9f P e253c5347150417495a6b6b50d19524b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e253c5347150417495a6b6b50d19524b, State: Initialized, Role: FOLLOWER
I20260809 15:12:04.549037 22113 ts_tablet_manager.cc:1428] T 4a89276adf354864a22670c0805a2a9f P a42c0ab1eca8460bb35b3717efdbc76a: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260809 15:12:04.549001 22115 consensus_queue.cc:260] T 4a89276adf354864a22670c0805a2a9f P e253c5347150417495a6b6b50d19524b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e253c5347150417495a6b6b50d19524b" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 41387 } } peers { permanent_uuid: "43c3ef5a5b8c478698f66060542192fd" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 42555 } } peers { permanent_uuid: "a42c0ab1eca8460bb35b3717efdbc76a" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 43433 } }
I20260809 15:12:04.550064 22089 heartbeater.cc:503] Master 127.24.183.254:38793 was elected leader, sending a full tablet report...
I20260809 15:12:04.550490 22115 ts_tablet_manager.cc:1428] T 4a89276adf354864a22670c0805a2a9f P e253c5347150417495a6b6b50d19524b: Time spent starting tablet: real 0.013s user 0.012s sys 0.001s
I20260809 15:12:04.553256 22120 raft_consensus.cc:482] T 4a89276adf354864a22670c0805a2a9f P e253c5347150417495a6b6b50d19524b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:12:04.553498 22120 raft_consensus.cc:504] T 4a89276adf354864a22670c0805a2a9f P e253c5347150417495a6b6b50d19524b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e253c5347150417495a6b6b50d19524b" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 41387 } } peers { permanent_uuid: "43c3ef5a5b8c478698f66060542192fd" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 42555 } } peers { permanent_uuid: "a42c0ab1eca8460bb35b3717efdbc76a" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 43433 } }
I20260809 15:12:04.555205 22120 leader_election.cc:288] T 4a89276adf354864a22670c0805a2a9f P e253c5347150417495a6b6b50d19524b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43c3ef5a5b8c478698f66060542192fd (127.24.183.194:42555), a42c0ab1eca8460bb35b3717efdbc76a (127.24.183.193:43433)
I20260809 15:12:04.561628 22092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:04.555616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:42555 (local address 127.24.183.195:32883)
0809 15:12:04.555996 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:04.556018 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:04.556160 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:04.556547 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:04.556550 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:04.556573 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:12:04.556770 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:04.556782 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:04.557908 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:04.557914 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:04.559593 (+ 1679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:04.559608 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:04.560987 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:04.560993 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:04.561153 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:04.561190 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 15:12:04.561272 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 15:12:04.561405 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244}
I20260809 15:12:04.561774 22109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:04.555595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:32883 (local address 127.24.183.194:42555)
0809 15:12:04.555801 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:04.555807 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:04.556092 (+ 285us) server_negotiation.cc:407] Connection header received
0809 15:12:04.556324 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:04.556329 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:04.556418 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:04.556503 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:12:04.556914 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:04.557776 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:04.559855 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:04.560837 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:04.560950 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:04.560998 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:04.561343 (+ 345us) server_negotiation.cc:299] Negotiation successful
0809 15:12:04.561485 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 15:12:04.562014 22110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:04.555620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:52233 (local address 127.24.183.193:43433)
0809 15:12:04.555832 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:04.555838 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:04.556417 (+ 579us) server_negotiation.cc:407] Connection header received
0809 15:12:04.556641 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:04.556646 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:04.556732 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:04.556828 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:12:04.557364 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:04.558125 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:04.560242 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:04.561111 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:04.561262 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:04.561324 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:04.561690 (+ 366us) server_negotiation.cc:299] Negotiation successful
0809 15:12:04.561813 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260809 15:12:04.562278 22123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:04.555692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:43433 (local address 127.24.183.195:52233)
0809 15:12:04.556320 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:04.556337 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:04.556461 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:04.556846 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:04.556851 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:04.556877 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:12:04.557235 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:04.557246 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:04.558275 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:04.558282 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:04.560074 (+ 1792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:04.560088 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:04.561310 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:04.561318 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:04.561511 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:04.561548 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 15:12:04.561662 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 15:12:04.561802 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":316,"threads_started":1}
I20260809 15:12:04.563087 21794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a89276adf354864a22670c0805a2a9f" candidate_uuid: "e253c5347150417495a6b6b50d19524b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43c3ef5a5b8c478698f66060542192fd" is_pre_election: true
I20260809 15:12:04.563330 21572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a89276adf354864a22670c0805a2a9f" candidate_uuid: "e253c5347150417495a6b6b50d19524b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a42c0ab1eca8460bb35b3717efdbc76a" is_pre_election: true
I20260809 15:12:04.563529 21794 raft_consensus.cc:2454] T 4a89276adf354864a22670c0805a2a9f P 43c3ef5a5b8c478698f66060542192fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e253c5347150417495a6b6b50d19524b in term 0.
I20260809 15:12:04.563917 21572 raft_consensus.cc:2454] T 4a89276adf354864a22670c0805a2a9f P a42c0ab1eca8460bb35b3717efdbc76a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e253c5347150417495a6b6b50d19524b in term 0.
I20260809 15:12:04.564304 21910 leader_election.cc:302] T 4a89276adf354864a22670c0805a2a9f P e253c5347150417495a6b6b50d19524b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43c3ef5a5b8c478698f66060542192fd, e253c5347150417495a6b6b50d19524b; no voters:
I20260809 15:12:04.564682 22120 raft_consensus.cc:2781] T 4a89276adf354864a22670c0805a2a9f P e253c5347150417495a6b6b50d19524b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:12:04.564805 22120 raft_consensus.cc:482] T 4a89276adf354864a22670c0805a2a9f P e253c5347150417495a6b6b50d19524b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:12:04.564913 22120 raft_consensus.cc:3037] T 4a89276adf354864a22670c0805a2a9f P e253c5347150417495a6b6b50d19524b [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:04.567062 22120 raft_consensus.cc:504] T 4a89276adf354864a22670c0805a2a9f P e253c5347150417495a6b6b50d19524b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e253c5347150417495a6b6b50d19524b" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 41387 } } peers { permanent_uuid: "43c3ef5a5b8c478698f66060542192fd" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 42555 } } peers { permanent_uuid: "a42c0ab1eca8460bb35b3717efdbc76a" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 43433 } }
I20260809 15:12:04.567586 22120 leader_election.cc:288] T 4a89276adf354864a22670c0805a2a9f P e253c5347150417495a6b6b50d19524b [CANDIDATE]: Term 1 election: Requested vote from peers 43c3ef5a5b8c478698f66060542192fd (127.24.183.194:42555), a42c0ab1eca8460bb35b3717efdbc76a (127.24.183.193:43433)
I20260809 15:12:04.568041 21572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a89276adf354864a22670c0805a2a9f" candidate_uuid: "e253c5347150417495a6b6b50d19524b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a42c0ab1eca8460bb35b3717efdbc76a"
I20260809 15:12:04.568081 21794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a89276adf354864a22670c0805a2a9f" candidate_uuid: "e253c5347150417495a6b6b50d19524b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43c3ef5a5b8c478698f66060542192fd"
I20260809 15:12:04.568203 21572 raft_consensus.cc:3037] T 4a89276adf354864a22670c0805a2a9f P a42c0ab1eca8460bb35b3717efdbc76a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:04.568302 21794 raft_consensus.cc:3037] T 4a89276adf354864a22670c0805a2a9f P 43c3ef5a5b8c478698f66060542192fd [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:04.570394 21572 raft_consensus.cc:2454] T 4a89276adf354864a22670c0805a2a9f P a42c0ab1eca8460bb35b3717efdbc76a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e253c5347150417495a6b6b50d19524b in term 1.
I20260809 15:12:04.570742 21907 leader_election.cc:302] T 4a89276adf354864a22670c0805a2a9f P e253c5347150417495a6b6b50d19524b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a42c0ab1eca8460bb35b3717efdbc76a, e253c5347150417495a6b6b50d19524b; no voters:
I20260809 15:12:04.570801 21794 raft_consensus.cc:2454] T 4a89276adf354864a22670c0805a2a9f P 43c3ef5a5b8c478698f66060542192fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e253c5347150417495a6b6b50d19524b in term 1.
I20260809 15:12:04.571019 22120 raft_consensus.cc:2781] T 4a89276adf354864a22670c0805a2a9f P e253c5347150417495a6b6b50d19524b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:12:04.571504 22120 raft_consensus.cc:686] T 4a89276adf354864a22670c0805a2a9f P e253c5347150417495a6b6b50d19524b [term 1 LEADER]: Becoming Leader. State: Replica: e253c5347150417495a6b6b50d19524b, State: Running, Role: LEADER
I20260809 15:12:04.572026 22120 consensus_queue.cc:237] T 4a89276adf354864a22670c0805a2a9f P e253c5347150417495a6b6b50d19524b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e253c5347150417495a6b6b50d19524b" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 41387 } } peers { permanent_uuid: "43c3ef5a5b8c478698f66060542192fd" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 42555 } } peers { permanent_uuid: "a42c0ab1eca8460bb35b3717efdbc76a" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 43433 } }
I20260809 15:12:04.576530 21344 catalog_manager.cc:5495] T 4a89276adf354864a22670c0805a2a9f P e253c5347150417495a6b6b50d19524b reported cstate change: term changed from 0 to 1, leader changed from <none> to e253c5347150417495a6b6b50d19524b (127.24.183.195). New cstate: current_term: 1 leader_uuid: "e253c5347150417495a6b6b50d19524b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e253c5347150417495a6b6b50d19524b" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 41387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43c3ef5a5b8c478698f66060542192fd" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 42555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a42c0ab1eca8460bb35b3717efdbc76a" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 43433 } health_report { overall_health: UNKNOWN } } }
I20260809 15:12:04.606215 22108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:04.601428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38042 (local address 127.24.183.195:41387)
0809 15:12:04.601601 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:04.601605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:04.601624 (+ 19us) server_negotiation.cc:407] Connection header received
0809 15:12:04.601760 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:04.601764 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:04.601844 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:04.601992 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:12:04.602282 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:04.602972 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:04.604373 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:04.604785 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:04.604872 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:04.605268 (+ 396us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:12:04.605766 (+ 498us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:12:04.605869 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:04.605909 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 15:12:04.605972 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 15:12:04.614123 21794 raft_consensus.cc:1261] T 4a89276adf354864a22670c0805a2a9f P 43c3ef5a5b8c478698f66060542192fd [term 1 FOLLOWER]: Refusing update from remote peer e253c5347150417495a6b6b50d19524b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:12:04.614123 21572 raft_consensus.cc:1261] T 4a89276adf354864a22670c0805a2a9f P a42c0ab1eca8460bb35b3717efdbc76a [term 1 FOLLOWER]: Refusing update from remote peer e253c5347150417495a6b6b50d19524b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:12:04.615128 22126 consensus_queue.cc:1035] T 4a89276adf354864a22670c0805a2a9f P e253c5347150417495a6b6b50d19524b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a42c0ab1eca8460bb35b3717efdbc76a" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 43433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:12:04.615484 22120 consensus_queue.cc:1035] T 4a89276adf354864a22670c0805a2a9f P e253c5347150417495a6b6b50d19524b [LEADER]: Connected to new peer: Peer: permanent_uuid: "43c3ef5a5b8c478698f66060542192fd" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 42555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:12:04.641031 21341 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46118:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:12:04.641355 21341 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46118:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:12:04.645385 21341 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a1b3c3071ad3496e9b6212d9b7aa9d72: Sending DeleteTablet for 3 replicas of tablet 4a89276adf354864a22670c0805a2a9f
I20260809 15:12:04.646637 21981 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a89276adf354864a22670c0805a2a9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:12:04 UTC) from {username='slave'} at 127.0.0.1:38026
I20260809 15:12:04.646670 21537 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a89276adf354864a22670c0805a2a9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:12:04 UTC) from {username='slave'} at 127.0.0.1:36090
I20260809 15:12:04.647320 21764 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a89276adf354864a22670c0805a2a9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:12:04 UTC) from {username='slave'} at 127.0.0.1:49496
I20260809 15:12:04.647480 22147 tablet_replica.cc:330] T 4a89276adf354864a22670c0805a2a9f P e253c5347150417495a6b6b50d19524b: stopping tablet replica
I20260809 15:12:04.647923 22147 raft_consensus.cc:2229] T 4a89276adf354864a22670c0805a2a9f P e253c5347150417495a6b6b50d19524b [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:12:04.648706 22147 raft_consensus.cc:2258] T 4a89276adf354864a22670c0805a2a9f P e253c5347150417495a6b6b50d19524b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:12:04.649559 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 21442
I20260809 15:12:04.650934 22147 ts_tablet_manager.cc:1905] T 4a89276adf354864a22670c0805a2a9f P e253c5347150417495a6b6b50d19524b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:12:04.658560 22147 ts_tablet_manager.cc:1918] T 4a89276adf354864a22670c0805a2a9f P e253c5347150417495a6b6b50d19524b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:12:04.658816 22147 log.cc:1196] T 4a89276adf354864a22670c0805a2a9f P e253c5347150417495a6b6b50d19524b: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-2/wal/wals/4a89276adf354864a22670c0805a2a9f
I20260809 15:12:04.659482 22147 ts_tablet_manager.cc:1939] T 4a89276adf354864a22670c0805a2a9f P e253c5347150417495a6b6b50d19524b: Deleting consensus metadata
I20260809 15:12:04.661401 21316 catalog_manager.cc:4841] TS e253c5347150417495a6b6b50d19524b (127.24.183.195:41387): tablet 4a89276adf354864a22670c0805a2a9f (table test-table [id=5284316b7c264ba085a7848030deb945]) successfully deleted
I20260809 15:12:04.664816 22150 tablet_replica.cc:330] T 4a89276adf354864a22670c0805a2a9f P 43c3ef5a5b8c478698f66060542192fd: stopping tablet replica
I20260809 15:12:04.665345 22150 raft_consensus.cc:2229] T 4a89276adf354864a22670c0805a2a9f P 43c3ef5a5b8c478698f66060542192fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:12:04.665934 22150 raft_consensus.cc:2258] T 4a89276adf354864a22670c0805a2a9f P 43c3ef5a5b8c478698f66060542192fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:12:04.666790 22148 tablet_replica.cc:330] T 4a89276adf354864a22670c0805a2a9f P a42c0ab1eca8460bb35b3717efdbc76a: stopping tablet replica
I20260809 15:12:04.667236 22148 raft_consensus.cc:2229] T 4a89276adf354864a22670c0805a2a9f P a42c0ab1eca8460bb35b3717efdbc76a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:12:04.667801 22148 raft_consensus.cc:2258] T 4a89276adf354864a22670c0805a2a9f P a42c0ab1eca8460bb35b3717efdbc76a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:12:04.668732 22150 ts_tablet_manager.cc:1905] T 4a89276adf354864a22670c0805a2a9f P 43c3ef5a5b8c478698f66060542192fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:12:04.669641 22148 ts_tablet_manager.cc:1905] T 4a89276adf354864a22670c0805a2a9f P a42c0ab1eca8460bb35b3717efdbc76a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:12:04.673296 22110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:04.652342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36098 (local address 127.24.183.193:43433)
0809 15:12:04.652521 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:04.652527 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:04.664178 (+ 11651us) server_negotiation.cc:407] Connection header received
0809 15:12:04.664362 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:04.664368 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:04.664478 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:04.664585 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:04.665276 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:04.666432 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:04.669211 (+ 2779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:04.669888 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:04.670001 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:04.671461 (+ 1460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:04.671473 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:04.671490 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:04.672254 (+ 764us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:04.672310 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:04.672334 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:04.672419 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:04.672666 (+ 247us) server_negotiation.cc:299] Negotiation successful
0809 15:12:04.672789 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 15:12:04.674679 21633 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:12:04.677708 22150 ts_tablet_manager.cc:1918] T 4a89276adf354864a22670c0805a2a9f P 43c3ef5a5b8c478698f66060542192fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:12:04.677875 22150 log.cc:1196] T 4a89276adf354864a22670c0805a2a9f P 43c3ef5a5b8c478698f66060542192fd: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288021313853-25311-0/minicluster-data/ts-1/wal/wals/4a89276adf354864a22670c0805a2a9f
I20260809 15:12:04.678432 22150 ts_tablet_manager.cc:1939] T 4a89276adf354864a22670c0805a2a9f P 43c3ef5a5b8c478698f66060542192fd: Deleting consensus metadata
I20260809 15:12:04.680399 21318 catalog_manager.cc:4841] TS 43c3ef5a5b8c478698f66060542192fd (127.24.183.194:42555): tablet 4a89276adf354864a22670c0805a2a9f (table test-table [id=5284316b7c264ba085a7848030deb945]) successfully deleted
I20260809 15:12:04.768864 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 21442
W20260809 15:12:04.794093 21317 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.183.193:43433 (error 108)
I20260809 15:12:04.794981 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 21647
I20260809 15:12:04.796440 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:04.795830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:43433 (local address 127.0.0.1:55776)
0809 15:12:04.796113 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:04.796240 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
W20260809 15:12:04.796716 21317 catalog_manager.cc:4584] TS a42c0ab1eca8460bb35b3717efdbc76a (127.24.183.193:43433): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4a89276adf354864a22670c0805a2a9f: Network error: Client connection negotiation failed: client connection to 127.24.183.193:43433: connect: Connection refused (error 111)
I20260809 15:12:04.802349 22109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:04.795867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36978 (local address 127.24.183.194:42555)
0809 15:12:04.796053 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:04.796058 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:04.796303 (+ 245us) server_negotiation.cc:407] Connection header received
0809 15:12:04.796480 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:04.796484 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:04.796620 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:04.796720 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:04.797180 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:04.797924 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:04.799297 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:04.799846 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:04.799956 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:04.800999 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:04.801008 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:04.801023 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:04.801607 (+ 584us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:04.801656 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:04.801676 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:04.801780 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:04.801918 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 15:12:04.802035 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 15:12:04.803032 21867 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:12:04.850556 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:04.849956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:43433 (local address 127.0.0.1:55780)
0809 15:12:04.850221 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:04.850353 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260809 15:12:04.851832 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:04.851337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:43433 (local address 127.0.0.1:55782)
0809 15:12:04.851524 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:04.851625 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 15:12:04.894449 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 21647
I20260809 15:12:04.912779 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:04.912085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:43433 (local address 127.0.0.1:55790)
0809 15:12:04.912422 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:04.912541 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175}
I20260809 15:12:04.914170 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:04.913693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:43433 (local address 127.0.0.1:55794)
0809 15:12:04.913932 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:04.914026 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260809 15:12:04.918516 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 21875
I20260809 15:12:04.929513 22108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:04.923218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47546 (local address 127.24.183.195:41387)
0809 15:12:04.923405 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:04.923411 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:04.923498 (+ 87us) server_negotiation.cc:407] Connection header received
0809 15:12:04.923709 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:04.923714 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:04.923817 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:04.923911 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:04.924346 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:04.925141 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:04.926444 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:04.926915 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:04.927030 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:04.928088 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:04.928097 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:04.928114 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:04.928746 (+ 632us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:04.928803 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:04.928823 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:04.928913 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:04.929060 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 15:12:04.929169 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 15:12:04.930256 22082 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:12:04.995739 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:04.995136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:43433 (local address 127.0.0.1:55808)
0809 15:12:04.995373 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:04.995527 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260809 15:12:04.996774 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:04.996387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:43433 (local address 127.0.0.1:55822)
0809 15:12:04.996585 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:04.996657 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260809 15:12:05.054579 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 21875
I20260809 15:12:05.083854 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 21287
I20260809 15:12:05.084955 21392 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:12:05.197351 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:05.196648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:43433 (local address 127.0.0.1:55828)
0809 15:12:05.196907 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:05.197057 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260809 15:12:05.197543 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 21287
2026-08-09T15:12:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2817 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:12:05 dist-test-slave-nj21 krb5kdc[22161](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:12:05 dist-test-slave-nj21 krb5kdc[22161](info): set up 2 sockets
Aug 09 15:12:05 dist-test-slave-nj21 krb5kdc[22161](info): commencing operation
krb5kdc: starting...
W20260809 15:12:06.433908 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.003s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:12:06 dist-test-slave-nj21 krb5kdc[22161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:12:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:12:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
I20260809 15:12:06.718274 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:46307
--webserver_interface=127.24.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:44863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:46307
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:12:07.054926 22185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:07.055235 22185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:07.055378 22185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:07.062724 22185 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:12:07.062845 22185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:07.062927 22185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:07.063004 22185 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:12:07.063066 22185 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:12:07.072990 22185 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:44863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:46307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:46307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:07.074391 22185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:07.075747 22185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:07.087955 22192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:07.087986 22193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:07.088704 22196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:07.089113 22185 server_base.cc:1034] running on GCE node
I20260809 15:12:07.090078 22185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:07.091897 22185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:07.093137 22185 hybrid_clock.cc:648] HybridClock initialized: now 1786288327093041 us; error 71 us; skew 500 ppm
I20260809 15:12:07.093559 22185 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:12:07 dist-test-slave-nj21 krb5kdc[22161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288327, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:12:07.100111 22185 init.cc:377] Logged in from keytab as kudu/127.24.183.254@KRBTEST.COM (short username kudu)
I20260809 15:12:07.108024 22185 webserver.cc:460] Webserver started at http://127.24.183.254:44751/ using document root <none> and password file <none>
I20260809 15:12:07.108762 22185 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:07.108893 22185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:07.109215 22185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:07.112354 22185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "3cd0bc4b4d13450a94a411088844752c"
format_stamp: "Formatted at 2026-08-09 15:12:07 on dist-test-slave-nj21"
server_key: "3937c56310e2794e4b07cccb2921746d"
server_key_iv: "d1883a78f4061c877f6f2732a9271872"
server_key_version: "encryptionkey@0"
I20260809 15:12:07.113211 22185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "3cd0bc4b4d13450a94a411088844752c"
format_stamp: "Formatted at 2026-08-09 15:12:07 on dist-test-slave-nj21"
server_key: "3937c56310e2794e4b07cccb2921746d"
server_key_iv: "d1883a78f4061c877f6f2732a9271872"
server_key_version: "encryptionkey@0"
I20260809 15:12:07.119251 22185 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260809 15:12:07.122603 22204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:07.123924 22185 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:12:07.124089 22185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "3cd0bc4b4d13450a94a411088844752c"
format_stamp: "Formatted at 2026-08-09 15:12:07 on dist-test-slave-nj21"
server_key: "3937c56310e2794e4b07cccb2921746d"
server_key_iv: "d1883a78f4061c877f6f2732a9271872"
server_key_version: "encryptionkey@0"
I20260809 15:12:07.124274 22185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:07.148929 22185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:07.153019 22185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:07.153291 22185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:07.171260 22185 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:46307
I20260809 15:12:07.171267 22285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:46307 every 8 connection(s)
I20260809 15:12:07.172640 22185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:12:07.174299 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 22185
I20260809 15:12:07.174486 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:12:07.174940 25311 external_mini_cluster.cc:1428] Setting key 131def493ac85364612de6e1030b5e47
I20260809 15:12:07.176994 22286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 15:12:07 dist-test-slave-nj21 krb5kdc[22161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:12:07.190037 22286 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cd0bc4b4d13450a94a411088844752c: Bootstrap starting.
I20260809 15:12:07.194154 22286 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3cd0bc4b4d13450a94a411088844752c: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:07.195324 22286 log.cc:824] T 00000000000000000000000000000000 P 3cd0bc4b4d13450a94a411088844752c: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:07.198814 22286 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cd0bc4b4d13450a94a411088844752c: No bootstrap required, opened a new log
I20260809 15:12:07.199119 22292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:07.176687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46998 (local address 127.24.183.254:46307)
0809 15:12:07.177429 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:07.177440 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:07.177480 (+ 40us) server_negotiation.cc:407] Connection header received
0809 15:12:07.179178 (+ 1698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:07.179199 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:07.179554 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:07.179915 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:07.181720 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:07.183047 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:07.184868 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:07.185495 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:07.185623 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:07.190526 (+ 4903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:07.190537 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:07.190556 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:12:07.190607 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:07.194407 (+ 3800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:07.195245 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:07.195253 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:07.195262 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:07.195514 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:07.196056 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:07.196061 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:07.196064 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:07.196637 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:07.196934 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:07.197396 (+ 462us) server_negotiation.cc:299] Negotiation successful
0809 15:12:07.197831 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":343,"threads_started":1}
I20260809 15:12:07.207139 22286 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cd0bc4b4d13450a94a411088844752c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cd0bc4b4d13450a94a411088844752c" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 46307 } }
I20260809 15:12:07.207484 22286 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3cd0bc4b4d13450a94a411088844752c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:07.207559 22286 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cd0bc4b4d13450a94a411088844752c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cd0bc4b4d13450a94a411088844752c, State: Initialized, Role: FOLLOWER
I20260809 15:12:07.208091 22286 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cd0bc4b4d13450a94a411088844752c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cd0bc4b4d13450a94a411088844752c" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 46307 } }
I20260809 15:12:07.208298 22286 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3cd0bc4b4d13450a94a411088844752c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:12:07.208397 22286 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3cd0bc4b4d13450a94a411088844752c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:12:07.208525 22286 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3cd0bc4b4d13450a94a411088844752c [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:07.210881 22286 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3cd0bc4b4d13450a94a411088844752c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cd0bc4b4d13450a94a411088844752c" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 46307 } }
I20260809 15:12:07.211442 22286 leader_election.cc:302] T 00000000000000000000000000000000 P 3cd0bc4b4d13450a94a411088844752c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3cd0bc4b4d13450a94a411088844752c; no voters:
I20260809 15:12:07.212112 22286 leader_election.cc:288] T 00000000000000000000000000000000 P 3cd0bc4b4d13450a94a411088844752c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:12:07.212231 22298 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3cd0bc4b4d13450a94a411088844752c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:12:07.212883 22298 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3cd0bc4b4d13450a94a411088844752c [term 1 LEADER]: Becoming Leader. State: Replica: 3cd0bc4b4d13450a94a411088844752c, State: Running, Role: LEADER
I20260809 15:12:07.213454 22298 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3cd0bc4b4d13450a94a411088844752c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cd0bc4b4d13450a94a411088844752c" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 46307 } }
I20260809 15:12:07.214098 22286 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cd0bc4b4d13450a94a411088844752c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:12:07.216249 22299 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cd0bc4b4d13450a94a411088844752c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3cd0bc4b4d13450a94a411088844752c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cd0bc4b4d13450a94a411088844752c" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 46307 } } }
I20260809 15:12:07.216281 22300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cd0bc4b4d13450a94a411088844752c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cd0bc4b4d13450a94a411088844752c. Latest consensus state: current_term: 1 leader_uuid: "3cd0bc4b4d13450a94a411088844752c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cd0bc4b4d13450a94a411088844752c" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 46307 } } }
I20260809 15:12:07.216795 22299 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cd0bc4b4d13450a94a411088844752c [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:07.216795 22300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cd0bc4b4d13450a94a411088844752c [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:07.218586 22305 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:12:07.223869 22305 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:12:07.232208 22305 catalog_manager.cc:1347] Generated new cluster ID: f9e91418013b47c2978b9383c75b5e01
I20260809 15:12:07.232360 22305 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:12:07.280838 22305 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:12:07.282778 22305 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:12:07.296065 22305 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3cd0bc4b4d13450a94a411088844752c: Generated new TSK 0
I20260809 15:12:07.296860 22305 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab.
Entry for principal kudu/127.24.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab.
I20260809 15:12:07.366384 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.193:0
--local_ip_for_outbound_sockets=127.24.183.193
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:46307
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:44863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:12:07.701591 22323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:07.701923 22323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:07.702046 22323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:07.709502 22323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:07.709606 22323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:07.709754 22323 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.193
I20260809 15:12:07.719837 22323 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:44863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:46307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.193
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:07.721118 22323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:07.722640 22323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:07.735816 22329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:07.735802 22338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:07.735859 22331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:07.736662 22323 server_base.cc:1034] running on GCE node
I20260809 15:12:07.737229 22323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:07.738178 22323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:07.739382 22323 hybrid_clock.cc:648] HybridClock initialized: now 1786288327739337 us; error 34 us; skew 500 ppm
I20260809 15:12:07.739665 22323 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:12:07 dist-test-slave-nj21 krb5kdc[22161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288327, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:12:07.742753 22323 init.cc:377] Logged in from keytab as kudu/127.24.183.193@KRBTEST.COM (short username kudu)
I20260809 15:12:07.744199 22323 webserver.cc:460] Webserver started at http://127.24.183.193:42109/ using document root <none> and password file <none>
I20260809 15:12:07.744855 22323 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:07.744975 22323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:07.745311 22323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:07.748224 22323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-0/data/instance:
uuid: "f6a0168984604c159b1ce3573320761d"
format_stamp: "Formatted at 2026-08-09 15:12:07 on dist-test-slave-nj21"
server_key: "c4d852e9e2923e53eb494f8e415eac11"
server_key_iv: "067acadc028024f5c2d44db5dd9fad55"
server_key_version: "encryptionkey@0"
I20260809 15:12:07.748970 22323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance:
uuid: "f6a0168984604c159b1ce3573320761d"
format_stamp: "Formatted at 2026-08-09 15:12:07 on dist-test-slave-nj21"
server_key: "c4d852e9e2923e53eb494f8e415eac11"
server_key_iv: "067acadc028024f5c2d44db5dd9fad55"
server_key_version: "encryptionkey@0"
I20260809 15:12:07.754009 22323 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 15:12:07.757328 22351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:07.758558 22323 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 15:12:07.758715 22323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-0/wal
uuid: "f6a0168984604c159b1ce3573320761d"
format_stamp: "Formatted at 2026-08-09 15:12:07 on dist-test-slave-nj21"
server_key: "c4d852e9e2923e53eb494f8e415eac11"
server_key_iv: "067acadc028024f5c2d44db5dd9fad55"
server_key_version: "encryptionkey@0"
I20260809 15:12:07.758903 22323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:07.782037 22323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:07.786100 22323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:07.786369 22323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:07.787276 22323 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:12:07.788625 22323 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:12:07.788708 22323 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:07.788829 22323 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:12:07.788884 22323 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:07.819362 22323 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.193:39891
I20260809 15:12:07.819403 22514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.193:39891 every 8 connection(s)
I20260809 15:12:07.820676 22323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
I20260809 15:12:07.823647 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 22323
I20260809 15:12:07.823861 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance
I20260809 15:12:07.824419 25311 external_mini_cluster.cc:1428] Setting key eef278c3c8b81479c16365a46b74863b
Aug 09 15:12:07 dist-test-slave-nj21 krb5kdc[22161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288327, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:12:07.841542 22520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:07.823177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:35927 (local address 127.24.183.254:46307)
0809 15:12:07.823712 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:07.823718 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:07.824781 (+ 1063us) server_negotiation.cc:407] Connection header received
0809 15:12:07.826780 (+ 1999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:07.826784 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:07.826847 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:07.826933 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:07.828904 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:07.829715 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:07.831704 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:07.832115 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:07.832208 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:07.836072 (+ 3864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:07.836077 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:07.836080 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:12:07.836112 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:07.838413 (+ 2301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:07.839224 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:07.839229 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:07.839232 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:07.839355 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:07.839788 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:07.839792 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:07.839795 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:07.840116 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:07.840269 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:07.841071 (+ 802us) server_negotiation.cc:299] Negotiation successful
0809 15:12:07.841246 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":249,"threads_started":1}
I20260809 15:12:07.842559 22521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:07.823753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:46307 (local address 127.24.183.193:35927)
0809 15:12:07.824596 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:07.824655 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:07.826477 (+ 1822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:07.827198 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:07.827210 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:07.827938 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:12:07.828736 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:07.828763 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:07.829861 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:07.829866 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:07.831563 (+ 1697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:07.831578 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:07.832339 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:07.832346 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:07.832611 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:07.833256 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:12:07.833285 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:12:07.835865 (+ 2580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:12:07.838545 (+ 2680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:12:07.838553 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:12:07.838567 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:12:07.839124 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:12:07.839445 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:12:07.839449 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:12:07.839452 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:12:07.839698 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:12:07.840261 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:12:07.840270 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:12:07.840665 (+ 395us) client_negotiation.cc:769] Sending connection context
0809 15:12:07.840960 (+ 295us) client_negotiation.cc:240] Negotiation successful
0809 15:12:07.841475 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":309,"threads_started":1}
I20260809 15:12:07.844131 22517 heartbeater.cc:347] Connected to a master server at 127.24.183.254:46307
I20260809 15:12:07.844447 22517 heartbeater.cc:464] Registering TS with master...
I20260809 15:12:07.845296 22517 heartbeater.cc:511] Master 127.24.183.254:46307 requested a full tablet report, sending...
I20260809 15:12:07.847405 22237 ts_manager.cc:194] Registered new tserver with Master: f6a0168984604c159b1ce3573320761d (127.24.183.193:39891)
I20260809 15:12:07.848871 22237 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.193@KRBTEST.COM'} at 127.24.183.193:35927
WARNING: no policy specified for kudu/127.24.183.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab.
Entry for principal kudu/127.24.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab.
I20260809 15:12:07.892081 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.194:0
--local_ip_for_outbound_sockets=127.24.183.194
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:46307
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:44863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:12:08.225328 22528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:08.225611 22528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:08.225754 22528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:08.233006 22528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:08.233125 22528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:08.233287 22528 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.194
I20260809 15:12:08.243000 22528 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:44863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:46307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.194
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:08.244359 22528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:08.245664 22528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:08.258255 22534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:08.258538 22538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:08.258647 22536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:08.259579 22528 server_base.cc:1034] running on GCE node
I20260809 15:12:08.260207 22528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:08.261062 22528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:08.262254 22528 hybrid_clock.cc:648] HybridClock initialized: now 1786288328262215 us; error 28 us; skew 500 ppm
I20260809 15:12:08.262547 22528 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:12:08 dist-test-slave-nj21 krb5kdc[22161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288328, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:12:08.265563 22528 init.cc:377] Logged in from keytab as kudu/127.24.183.194@KRBTEST.COM (short username kudu)
I20260809 15:12:08.266906 22528 webserver.cc:460] Webserver started at http://127.24.183.194:34035/ using document root <none> and password file <none>
I20260809 15:12:08.267550 22528 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:08.267686 22528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:08.267997 22528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:08.270892 22528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-1/data/instance:
uuid: "d35b7a1a1def4fd39a7c727f5f4b8c81"
format_stamp: "Formatted at 2026-08-09 15:12:08 on dist-test-slave-nj21"
server_key: "83afbceb4f457127cd5cb2ca49150133"
server_key_iv: "61576465ddd81a4b7c44350e3fd0d38d"
server_key_version: "encryptionkey@0"
I20260809 15:12:08.271628 22528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance:
uuid: "d35b7a1a1def4fd39a7c727f5f4b8c81"
format_stamp: "Formatted at 2026-08-09 15:12:08 on dist-test-slave-nj21"
server_key: "83afbceb4f457127cd5cb2ca49150133"
server_key_iv: "61576465ddd81a4b7c44350e3fd0d38d"
server_key_version: "encryptionkey@0"
I20260809 15:12:08.276746 22528 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260809 15:12:08.280386 22548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:08.281630 22528 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:12:08.281787 22528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-1/wal
uuid: "d35b7a1a1def4fd39a7c727f5f4b8c81"
format_stamp: "Formatted at 2026-08-09 15:12:08 on dist-test-slave-nj21"
server_key: "83afbceb4f457127cd5cb2ca49150133"
server_key_iv: "61576465ddd81a4b7c44350e3fd0d38d"
server_key_version: "encryptionkey@0"
I20260809 15:12:08.281966 22528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:08.299487 22528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:08.303258 22528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:08.303534 22528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:08.304561 22528 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:12:08.305891 22528 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:12:08.305969 22528 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:08.306097 22528 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:12:08.306151 22528 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:08.336570 22528 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.194:37189
I20260809 15:12:08.336616 22723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.194:37189 every 8 connection(s)
I20260809 15:12:08.337834 22528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
I20260809 15:12:08.339020 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 22528
I20260809 15:12:08.339196 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance
I20260809 15:12:08.339788 25311 external_mini_cluster.cc:1428] Setting key a98596c1656f5b0de77698e0633f2b19
Aug 09 15:12:08 dist-test-slave-nj21 krb5kdc[22161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288328, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:12:08.359598 22520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:08.340575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:53083 (local address 127.24.183.254:46307)
0809 15:12:08.340961 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:08.340966 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:08.342076 (+ 1110us) server_negotiation.cc:407] Connection header received
0809 15:12:08.344611 (+ 2535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:08.344616 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:08.344683 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:08.344766 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:08.346689 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:08.347413 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:08.349393 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:08.349825 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:08.349916 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:08.353802 (+ 3886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:08.353807 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:08.353810 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:12:08.353844 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:08.356750 (+ 2906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:08.357421 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:08.357425 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:08.357427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:08.357552 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:08.357957 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:08.357960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:08.357962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:08.358229 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:08.358364 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:08.359028 (+ 664us) server_negotiation.cc:299] Negotiation successful
0809 15:12:08.359263 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274}
I20260809 15:12:08.360569 22728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:08.341122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:46307 (local address 127.24.183.194:53083)
0809 15:12:08.341911 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:08.341950 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:08.344315 (+ 2365us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:08.345058 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:08.345067 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:08.345741 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:12:08.346507 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:08.346532 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:08.347568 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:08.347572 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:08.349273 (+ 1701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:08.349288 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:08.350046 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:08.350053 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:08.350312 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:08.350975 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:12:08.351011 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:12:08.353575 (+ 2564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:12:08.356874 (+ 3299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:12:08.356880 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:12:08.356892 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:12:08.357284 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:12:08.357655 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:12:08.357660 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:12:08.357664 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:12:08.357835 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:12:08.358353 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:12:08.358360 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:12:08.358671 (+ 311us) client_negotiation.cc:769] Sending connection context
0809 15:12:08.358897 (+ 226us) client_negotiation.cc:240] Negotiation successful
0809 15:12:08.359430 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":301,"threads_started":1}
I20260809 15:12:08.362052 22724 heartbeater.cc:347] Connected to a master server at 127.24.183.254:46307
I20260809 15:12:08.362335 22724 heartbeater.cc:464] Registering TS with master...
I20260809 15:12:08.363032 22724 heartbeater.cc:511] Master 127.24.183.254:46307 requested a full tablet report, sending...
I20260809 15:12:08.364627 22237 ts_manager.cc:194] Registered new tserver with Master: d35b7a1a1def4fd39a7c727f5f4b8c81 (127.24.183.194:37189)
I20260809 15:12:08.365828 22237 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.194@KRBTEST.COM'} at 127.24.183.194:53083
WARNING: no policy specified for kudu/127.24.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab.
Entry for principal kudu/127.24.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab.
I20260809 15:12:08.406407 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.195:0
--local_ip_for_outbound_sockets=127.24.183.195
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:46307
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:44863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:12:08.742304 22732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:08.742578 22732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:08.742709 22732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:08.750326 22732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:08.750438 22732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:08.750595 22732 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.195
I20260809 15:12:08.760584 22732 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:44863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:46307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.195
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:08.761904 22732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:08.763262 22732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:08.776175 22740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:08.776160 22741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:08.776135 22744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:08.776863 22732 server_base.cc:1034] running on GCE node
I20260809 15:12:08.777431 22732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:08.778350 22732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:08.779601 22732 hybrid_clock.cc:648] HybridClock initialized: now 1786288328779524 us; error 48 us; skew 500 ppm
I20260809 15:12:08.779951 22732 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:12:08 dist-test-slave-nj21 krb5kdc[22161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288328, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:12:08.783007 22732 init.cc:377] Logged in from keytab as kudu/127.24.183.195@KRBTEST.COM (short username kudu)
I20260809 15:12:08.784476 22732 webserver.cc:460] Webserver started at http://127.24.183.195:34379/ using document root <none> and password file <none>
I20260809 15:12:08.785156 22732 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:08.785285 22732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:08.785585 22732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:08.788487 22732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-2/data/instance:
uuid: "fe558a4930c8421e9698fa907059c9fb"
format_stamp: "Formatted at 2026-08-09 15:12:08 on dist-test-slave-nj21"
server_key: "a6bbb6cb0692c377597d26a06d8f5c85"
server_key_iv: "27274b4b55bedda22767f71f56e0ad18"
server_key_version: "encryptionkey@0"
I20260809 15:12:08.789239 22732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance:
uuid: "fe558a4930c8421e9698fa907059c9fb"
format_stamp: "Formatted at 2026-08-09 15:12:08 on dist-test-slave-nj21"
server_key: "a6bbb6cb0692c377597d26a06d8f5c85"
server_key_iv: "27274b4b55bedda22767f71f56e0ad18"
server_key_version: "encryptionkey@0"
I20260809 15:12:08.794219 22732 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 15:12:08.797498 22755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:08.798743 22732 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:12:08.798894 22732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-2/wal
uuid: "fe558a4930c8421e9698fa907059c9fb"
format_stamp: "Formatted at 2026-08-09 15:12:08 on dist-test-slave-nj21"
server_key: "a6bbb6cb0692c377597d26a06d8f5c85"
server_key_iv: "27274b4b55bedda22767f71f56e0ad18"
server_key_version: "encryptionkey@0"
I20260809 15:12:08.799067 22732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:08.812808 22732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:08.816830 22732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:08.817095 22732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:08.818010 22732 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:12:08.819444 22732 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:12:08.819517 22732 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:08.819633 22732 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:12:08.819764 22732 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260809 15:12:08.851037 22732 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.195:38397
I20260809 15:12:08.851075 22912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.195:38397 every 8 connection(s)
I20260809 15:12:08.852353 22732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
I20260809 15:12:08.852344 22517 heartbeater.cc:503] Master 127.24.183.254:46307 was elected leader, sending a full tablet report...
I20260809 15:12:08.862066 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 22732
I20260809 15:12:08.862258 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance
I20260809 15:12:08.862845 25311 external_mini_cluster.cc:1428] Setting key 8c919ce12cb8e95d73570c8a47a576af
Aug 09 15:12:08 dist-test-slave-nj21 krb5kdc[22161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288328, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:12:08.871565 22520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:08.854983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:49795 (local address 127.24.183.254:46307)
0809 15:12:08.855185 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:08.855191 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:08.856340 (+ 1149us) server_negotiation.cc:407] Connection header received
0809 15:12:08.858135 (+ 1795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:08.858139 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:08.858208 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:08.858289 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:08.859730 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:08.860483 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:08.862018 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:08.862680 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:08.862814 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:08.866528 (+ 3714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:08.866533 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:08.866536 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:12:08.866567 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:08.868933 (+ 2366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:08.869616 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:08.869620 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:08.869622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:08.869740 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:08.870132 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:08.870135 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:08.870137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:08.870400 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:08.870507 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:08.871154 (+ 647us) server_negotiation.cc:299] Negotiation successful
0809 15:12:08.871312 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 15:12:08.872545 22918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:08.855415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:46307 (local address 127.24.183.195:49795)
0809 15:12:08.856176 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:08.856225 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:08.857866 (+ 1641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:08.858516 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:08.858524 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:08.859026 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:12:08.859571 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:08.859588 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:08.860607 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:08.860611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:08.861877 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:08.861889 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:08.862902 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:08.862909 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:08.863218 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:08.863994 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:12:08.864019 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:12:08.866344 (+ 2325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:12:08.869068 (+ 2724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:12:08.869075 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:12:08.869085 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:12:08.869526 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:12:08.869845 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:12:08.869848 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:12:08.869851 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:12:08.870047 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:12:08.870505 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:12:08.870511 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:12:08.870851 (+ 340us) client_negotiation.cc:769] Sending connection context
0809 15:12:08.871086 (+ 235us) client_negotiation.cc:240] Negotiation successful
0809 15:12:08.871466 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":319,"threads_started":1}
I20260809 15:12:08.873771 22913 heartbeater.cc:347] Connected to a master server at 127.24.183.254:46307
I20260809 15:12:08.874043 22913 heartbeater.cc:464] Registering TS with master...
I20260809 15:12:08.874742 22913 heartbeater.cc:511] Master 127.24.183.254:46307 requested a full tablet report, sending...
I20260809 15:12:08.876075 22237 ts_manager.cc:194] Registered new tserver with Master: fe558a4930c8421e9698fa907059c9fb (127.24.183.195:38397)
I20260809 15:12:08.877115 22237 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.195@KRBTEST.COM'} at 127.24.183.195:49795
I20260809 15:12:08.877861 25311 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:12:08.895129 22520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:08.882417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47006 (local address 127.24.183.254:46307)
0809 15:12:08.882608 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:08.882615 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:08.883003 (+ 388us) server_negotiation.cc:407] Connection header received
0809 15:12:08.883114 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:08.883118 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:08.883175 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:08.883262 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:08.884830 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:08.885622 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:08.887021 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:08.887435 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:08.887521 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:08.890082 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:08.890087 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:08.890090 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:12:08.890121 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:08.892520 (+ 2399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:08.893294 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:08.893297 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:08.893300 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:08.893425 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:08.893814 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:08.893818 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:08.893820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:08.894125 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:08.894350 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:08.894661 (+ 311us) server_negotiation.cc:299] Negotiation successful
0809 15:12:08.894794 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 15:12:08.898903 22235 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47006:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:12:08.905915 22235 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:12:08.930112 22934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:08.917942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:37189 (local address 127.0.0.1:54668)
0809 15:12:08.919535 (+ 1593us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:08.919552 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:08.919655 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:08.921018 (+ 1363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:08.921026 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:08.921059 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:12:08.921582 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:08.921594 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:08.924911 (+ 3317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:08.924916 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:08.927215 (+ 2299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:08.927230 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:08.929086 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:08.929093 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:08.929288 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:08.929338 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 15:12:08.929449 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 15:12:08.929623 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1471,"mutex_wait_us":8,"thread_start_us":1319,"threads_started":1}
I20260809 15:12:08.931308 22938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:08.918456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54668 (local address 127.24.183.194:37189)
0809 15:12:08.920365 (+ 1909us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:08.920374 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:08.920403 (+ 29us) server_negotiation.cc:407] Connection header received
0809 15:12:08.920508 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:08.920516 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:08.920765 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:08.921805 (+ 1040us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:12:08.922056 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:08.923644 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:08.927419 (+ 3775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:08.928852 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:08.930595 (+ 1743us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:08.930768 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:08.930835 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 15:12:08.931015 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1722,"thread_start_us":1492,"threads_started":1}
I20260809 15:12:08.933840 22933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:08.917786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:39891 (local address 127.0.0.1:59276)
0809 15:12:08.918706 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:08.918746 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:08.918943 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:08.924556 (+ 5613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:08.924560 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:08.924585 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:12:08.924989 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:08.925000 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:08.928031 (+ 3031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:08.928036 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:08.930129 (+ 2093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:08.930143 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:08.933070 (+ 2927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:08.933076 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:08.933249 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:08.933280 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 15:12:08.933346 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 15:12:08.933482 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"thread_start_us":1009,"threads_started":1}
I20260809 15:12:08.934952 22935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:08.918099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59276 (local address 127.24.183.193:39891)
0809 15:12:08.924147 (+ 6048us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:08.924153 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:08.924174 (+ 21us) server_negotiation.cc:407] Connection header received
0809 15:12:08.924244 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:08.924249 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:08.924391 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:08.924567 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:12:08.926338 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:08.927888 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:08.931339 (+ 3451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:08.932825 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:08.934248 (+ 1423us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:08.934436 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:08.934504 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 15:12:08.934648 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5892,"thread_start_us":5737,"threads_started":1}
I20260809 15:12:08.937757 22422 tablet_service.cc:1467] Processing CreateTablet for tablet 7f12d76f888b4812ac534249a705ce76 (DEFAULT_TABLE table=test-table [id=538ba344d9da443587a93843df68bbbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:12:08.935559 22619 tablet_service.cc:1467] Processing CreateTablet for tablet 7f12d76f888b4812ac534249a705ce76 (DEFAULT_TABLE table=test-table [id=538ba344d9da443587a93843df68bbbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:12:08.939029 22934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:08.920088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:38397 (local address 127.0.0.1:34074)
0809 15:12:08.930322 (+ 10234us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:08.930337 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:08.930420 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:08.933862 (+ 3442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:08.933867 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:08.933903 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:12:08.934074 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:08.934084 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:08.935868 (+ 1784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:08.935873 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:08.937111 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:08.937123 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:08.938301 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:08.938307 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:08.938504 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:08.938534 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 15:12:08.938631 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 15:12:08.938761 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10156,"thread_start_us":13542,"threads_started":1}
I20260809 15:12:08.939165 22422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f12d76f888b4812ac534249a705ce76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:08.939175 22940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:08.920548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34074 (local address 127.24.183.195:38397)
0809 15:12:08.933000 (+ 12452us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:08.933007 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:08.933037 (+ 30us) server_negotiation.cc:407] Connection header received
0809 15:12:08.933129 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:08.933137 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:08.933348 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:08.934234 (+ 886us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:12:08.934455 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:08.935703 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:08.937275 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:08.938137 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:08.938291 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:08.938452 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:08.938701 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 15:12:08.938859 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12251,"thread_start_us":13231,"threads_started":1}
I20260809 15:12:08.939793 22619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f12d76f888b4812ac534249a705ce76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:08.942696 22826 tablet_service.cc:1467] Processing CreateTablet for tablet 7f12d76f888b4812ac534249a705ce76 (DEFAULT_TABLE table=test-table [id=538ba344d9da443587a93843df68bbbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:12:08.944124 22826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f12d76f888b4812ac534249a705ce76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:08.951366 22941 tablet_bootstrap.cc:492] T 7f12d76f888b4812ac534249a705ce76 P d35b7a1a1def4fd39a7c727f5f4b8c81: Bootstrap starting.
I20260809 15:12:08.953423 22944 tablet_bootstrap.cc:492] T 7f12d76f888b4812ac534249a705ce76 P f6a0168984604c159b1ce3573320761d: Bootstrap starting.
I20260809 15:12:08.954905 22941 tablet_bootstrap.cc:654] T 7f12d76f888b4812ac534249a705ce76 P d35b7a1a1def4fd39a7c727f5f4b8c81: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:08.956219 22941 log.cc:824] T 7f12d76f888b4812ac534249a705ce76 P d35b7a1a1def4fd39a7c727f5f4b8c81: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:08.956900 22944 tablet_bootstrap.cc:654] T 7f12d76f888b4812ac534249a705ce76 P f6a0168984604c159b1ce3573320761d: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:08.957180 22945 tablet_bootstrap.cc:492] T 7f12d76f888b4812ac534249a705ce76 P fe558a4930c8421e9698fa907059c9fb: Bootstrap starting.
I20260809 15:12:08.958582 22944 log.cc:824] T 7f12d76f888b4812ac534249a705ce76 P f6a0168984604c159b1ce3573320761d: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:08.960426 22941 tablet_bootstrap.cc:492] T 7f12d76f888b4812ac534249a705ce76 P d35b7a1a1def4fd39a7c727f5f4b8c81: No bootstrap required, opened a new log
I20260809 15:12:08.960461 22945 tablet_bootstrap.cc:654] T 7f12d76f888b4812ac534249a705ce76 P fe558a4930c8421e9698fa907059c9fb: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:08.960695 22941 ts_tablet_manager.cc:1397] T 7f12d76f888b4812ac534249a705ce76 P d35b7a1a1def4fd39a7c727f5f4b8c81: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.001s
I20260809 15:12:08.961393 22945 log.cc:824] T 7f12d76f888b4812ac534249a705ce76 P fe558a4930c8421e9698fa907059c9fb: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:08.961512 22944 tablet_bootstrap.cc:492] T 7f12d76f888b4812ac534249a705ce76 P f6a0168984604c159b1ce3573320761d: No bootstrap required, opened a new log
I20260809 15:12:08.961818 22944 ts_tablet_manager.cc:1397] T 7f12d76f888b4812ac534249a705ce76 P f6a0168984604c159b1ce3573320761d: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 15:12:08.963927 22945 tablet_bootstrap.cc:492] T 7f12d76f888b4812ac534249a705ce76 P fe558a4930c8421e9698fa907059c9fb: No bootstrap required, opened a new log
I20260809 15:12:08.964145 22945 ts_tablet_manager.cc:1397] T 7f12d76f888b4812ac534249a705ce76 P fe558a4930c8421e9698fa907059c9fb: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260809 15:12:08.970847 22941 raft_consensus.cc:348] T 7f12d76f888b4812ac534249a705ce76 P d35b7a1a1def4fd39a7c727f5f4b8c81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6a0168984604c159b1ce3573320761d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39891 } } peers { permanent_uuid: "d35b7a1a1def4fd39a7c727f5f4b8c81" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 37189 } } peers { permanent_uuid: "fe558a4930c8421e9698fa907059c9fb" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38397 } }
I20260809 15:12:08.971299 22941 raft_consensus.cc:374] T 7f12d76f888b4812ac534249a705ce76 P d35b7a1a1def4fd39a7c727f5f4b8c81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:08.971437 22941 raft_consensus.cc:729] T 7f12d76f888b4812ac534249a705ce76 P d35b7a1a1def4fd39a7c727f5f4b8c81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35b7a1a1def4fd39a7c727f5f4b8c81, State: Initialized, Role: FOLLOWER
I20260809 15:12:08.971361 22945 raft_consensus.cc:348] T 7f12d76f888b4812ac534249a705ce76 P fe558a4930c8421e9698fa907059c9fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6a0168984604c159b1ce3573320761d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39891 } } peers { permanent_uuid: "d35b7a1a1def4fd39a7c727f5f4b8c81" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 37189 } } peers { permanent_uuid: "fe558a4930c8421e9698fa907059c9fb" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38397 } }
I20260809 15:12:08.971772 22945 raft_consensus.cc:374] T 7f12d76f888b4812ac534249a705ce76 P fe558a4930c8421e9698fa907059c9fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:08.971889 22945 raft_consensus.cc:729] T 7f12d76f888b4812ac534249a705ce76 P fe558a4930c8421e9698fa907059c9fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe558a4930c8421e9698fa907059c9fb, State: Initialized, Role: FOLLOWER
I20260809 15:12:08.972188 22941 consensus_queue.cc:260] T 7f12d76f888b4812ac534249a705ce76 P d35b7a1a1def4fd39a7c727f5f4b8c81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6a0168984604c159b1ce3573320761d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39891 } } peers { permanent_uuid: "d35b7a1a1def4fd39a7c727f5f4b8c81" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 37189 } } peers { permanent_uuid: "fe558a4930c8421e9698fa907059c9fb" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38397 } }
I20260809 15:12:08.972234 22944 raft_consensus.cc:348] T 7f12d76f888b4812ac534249a705ce76 P f6a0168984604c159b1ce3573320761d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6a0168984604c159b1ce3573320761d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39891 } } peers { permanent_uuid: "d35b7a1a1def4fd39a7c727f5f4b8c81" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 37189 } } peers { permanent_uuid: "fe558a4930c8421e9698fa907059c9fb" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38397 } }
I20260809 15:12:08.972402 22945 consensus_queue.cc:260] T 7f12d76f888b4812ac534249a705ce76 P fe558a4930c8421e9698fa907059c9fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6a0168984604c159b1ce3573320761d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39891 } } peers { permanent_uuid: "d35b7a1a1def4fd39a7c727f5f4b8c81" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 37189 } } peers { permanent_uuid: "fe558a4930c8421e9698fa907059c9fb" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38397 } }
I20260809 15:12:08.972647 22944 raft_consensus.cc:374] T 7f12d76f888b4812ac534249a705ce76 P f6a0168984604c159b1ce3573320761d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:08.972770 22944 raft_consensus.cc:729] T 7f12d76f888b4812ac534249a705ce76 P f6a0168984604c159b1ce3573320761d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6a0168984604c159b1ce3573320761d, State: Initialized, Role: FOLLOWER
I20260809 15:12:08.973399 22913 heartbeater.cc:503] Master 127.24.183.254:46307 was elected leader, sending a full tablet report...
I20260809 15:12:08.973502 22944 consensus_queue.cc:260] T 7f12d76f888b4812ac534249a705ce76 P f6a0168984604c159b1ce3573320761d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6a0168984604c159b1ce3573320761d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39891 } } peers { permanent_uuid: "d35b7a1a1def4fd39a7c727f5f4b8c81" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 37189 } } peers { permanent_uuid: "fe558a4930c8421e9698fa907059c9fb" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38397 } }
I20260809 15:12:08.975258 22945 ts_tablet_manager.cc:1428] T 7f12d76f888b4812ac534249a705ce76 P fe558a4930c8421e9698fa907059c9fb: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260809 15:12:08.976101 22724 heartbeater.cc:503] Master 127.24.183.254:46307 was elected leader, sending a full tablet report...
I20260809 15:12:08.976326 22944 ts_tablet_manager.cc:1428] T 7f12d76f888b4812ac534249a705ce76 P f6a0168984604c159b1ce3573320761d: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260809 15:12:08.976758 22941 ts_tablet_manager.cc:1428] T 7f12d76f888b4812ac534249a705ce76 P d35b7a1a1def4fd39a7c727f5f4b8c81: Time spent starting tablet: real 0.016s user 0.012s sys 0.003s
W20260809 15:12:09.073004 22518 tablet.cc:2369] T 7f12d76f888b4812ac534249a705ce76 P f6a0168984604c159b1ce3573320761d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:12:09.089771 22725 tablet.cc:2369] T 7f12d76f888b4812ac534249a705ce76 P d35b7a1a1def4fd39a7c727f5f4b8c81: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:12:09.092689 22951 raft_consensus.cc:482] T 7f12d76f888b4812ac534249a705ce76 P d35b7a1a1def4fd39a7c727f5f4b8c81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:12:09.093016 22951 raft_consensus.cc:504] T 7f12d76f888b4812ac534249a705ce76 P d35b7a1a1def4fd39a7c727f5f4b8c81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6a0168984604c159b1ce3573320761d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39891 } } peers { permanent_uuid: "d35b7a1a1def4fd39a7c727f5f4b8c81" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 37189 } } peers { permanent_uuid: "fe558a4930c8421e9698fa907059c9fb" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38397 } }
I20260809 15:12:09.094656 22951 leader_election.cc:288] T 7f12d76f888b4812ac534249a705ce76 P d35b7a1a1def4fd39a7c727f5f4b8c81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6a0168984604c159b1ce3573320761d (127.24.183.193:39891), fe558a4930c8421e9698fa907059c9fb (127.24.183.195:38397)
I20260809 15:12:09.101826 22935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:09.095114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:39851 (local address 127.24.183.193:39891)
0809 15:12:09.095342 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:09.095349 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:09.095846 (+ 497us) server_negotiation.cc:407] Connection header received
0809 15:12:09.096086 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:09.096092 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:09.096176 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:09.096273 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:12:09.096941 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:09.097859 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:09.099911 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:09.100859 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:09.101053 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:09.101147 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:09.101409 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 15:12:09.101553 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 15:12:09.101851 22940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:09.096126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:53255 (local address 127.24.183.195:38397)
0809 15:12:09.096300 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:09.096306 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:09.096556 (+ 250us) server_negotiation.cc:407] Connection header received
0809 15:12:09.096721 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:09.096726 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:09.096805 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:09.096908 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:12:09.097489 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:09.098362 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:09.100128 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:09.100888 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:09.101054 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:09.101133 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:09.101492 (+ 359us) server_negotiation.cc:299] Negotiation successful
0809 15:12:09.101624 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 15:12:09.102000 22955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:09.095054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:39891 (local address 127.24.183.194:39851)
0809 15:12:09.095733 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:09.095750 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:09.095927 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:09.096322 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:09.096327 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:09.096361 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:12:09.096787 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:09.096801 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:09.097984 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:09.097990 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:09.099751 (+ 1761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:09.099764 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:09.101077 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:09.101083 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:09.101258 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:09.101288 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 15:12:09.101360 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 15:12:09.101501 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"mutex_wait_us":56,"thread_start_us":363,"threads_started":1}
I20260809 15:12:09.102001 22956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:09.095869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:38397 (local address 127.24.183.194:53255)
0809 15:12:09.096461 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:09.096475 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:09.096592 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:09.096928 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:09.096932 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:09.096957 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:12:09.097323 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:09.097334 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:09.098499 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:09.098503 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:09.099978 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:09.099991 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:09.101172 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:09.101178 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:09.101324 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:09.101351 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 15:12:09.101449 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 15:12:09.101581 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"mutex_wait_us":2,"spinlock_wait_cycles":2304,"thread_start_us":411,"threads_started":1}
I20260809 15:12:09.103199 22450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f12d76f888b4812ac534249a705ce76" candidate_uuid: "d35b7a1a1def4fd39a7c727f5f4b8c81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6a0168984604c159b1ce3573320761d" is_pre_election: true
I20260809 15:12:09.103209 22852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f12d76f888b4812ac534249a705ce76" candidate_uuid: "d35b7a1a1def4fd39a7c727f5f4b8c81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe558a4930c8421e9698fa907059c9fb" is_pre_election: true
I20260809 15:12:09.103785 22450 raft_consensus.cc:2454] T 7f12d76f888b4812ac534249a705ce76 P f6a0168984604c159b1ce3573320761d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35b7a1a1def4fd39a7c727f5f4b8c81 in term 0.
I20260809 15:12:09.103754 22852 raft_consensus.cc:2454] T 7f12d76f888b4812ac534249a705ce76 P fe558a4930c8421e9698fa907059c9fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35b7a1a1def4fd39a7c727f5f4b8c81 in term 0.
W20260809 15:12:09.104023 22916 tablet.cc:2369] T 7f12d76f888b4812ac534249a705ce76 P fe558a4930c8421e9698fa907059c9fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:12:09.104516 22553 leader_election.cc:302] T 7f12d76f888b4812ac534249a705ce76 P d35b7a1a1def4fd39a7c727f5f4b8c81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d35b7a1a1def4fd39a7c727f5f4b8c81, fe558a4930c8421e9698fa907059c9fb; no voters:
I20260809 15:12:09.104938 22951 raft_consensus.cc:2781] T 7f12d76f888b4812ac534249a705ce76 P d35b7a1a1def4fd39a7c727f5f4b8c81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:12:09.105079 22951 raft_consensus.cc:482] T 7f12d76f888b4812ac534249a705ce76 P d35b7a1a1def4fd39a7c727f5f4b8c81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:12:09.105190 22951 raft_consensus.cc:3037] T 7f12d76f888b4812ac534249a705ce76 P d35b7a1a1def4fd39a7c727f5f4b8c81 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:09.107425 22951 raft_consensus.cc:504] T 7f12d76f888b4812ac534249a705ce76 P d35b7a1a1def4fd39a7c727f5f4b8c81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6a0168984604c159b1ce3573320761d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39891 } } peers { permanent_uuid: "d35b7a1a1def4fd39a7c727f5f4b8c81" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 37189 } } peers { permanent_uuid: "fe558a4930c8421e9698fa907059c9fb" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38397 } }
I20260809 15:12:09.108057 22951 leader_election.cc:288] T 7f12d76f888b4812ac534249a705ce76 P d35b7a1a1def4fd39a7c727f5f4b8c81 [CANDIDATE]: Term 1 election: Requested vote from peers f6a0168984604c159b1ce3573320761d (127.24.183.193:39891), fe558a4930c8421e9698fa907059c9fb (127.24.183.195:38397)
I20260809 15:12:09.108487 22450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f12d76f888b4812ac534249a705ce76" candidate_uuid: "d35b7a1a1def4fd39a7c727f5f4b8c81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6a0168984604c159b1ce3573320761d"
I20260809 15:12:09.108546 22852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f12d76f888b4812ac534249a705ce76" candidate_uuid: "d35b7a1a1def4fd39a7c727f5f4b8c81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe558a4930c8421e9698fa907059c9fb"
I20260809 15:12:09.108688 22450 raft_consensus.cc:3037] T 7f12d76f888b4812ac534249a705ce76 P f6a0168984604c159b1ce3573320761d [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:09.108734 22852 raft_consensus.cc:3037] T 7f12d76f888b4812ac534249a705ce76 P fe558a4930c8421e9698fa907059c9fb [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:09.111234 22852 raft_consensus.cc:2454] T 7f12d76f888b4812ac534249a705ce76 P fe558a4930c8421e9698fa907059c9fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35b7a1a1def4fd39a7c727f5f4b8c81 in term 1.
I20260809 15:12:09.111244 22450 raft_consensus.cc:2454] T 7f12d76f888b4812ac534249a705ce76 P f6a0168984604c159b1ce3573320761d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35b7a1a1def4fd39a7c727f5f4b8c81 in term 1.
I20260809 15:12:09.111605 22553 leader_election.cc:302] T 7f12d76f888b4812ac534249a705ce76 P d35b7a1a1def4fd39a7c727f5f4b8c81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d35b7a1a1def4fd39a7c727f5f4b8c81, f6a0168984604c159b1ce3573320761d; no voters:
I20260809 15:12:09.111927 22951 raft_consensus.cc:2781] T 7f12d76f888b4812ac534249a705ce76 P d35b7a1a1def4fd39a7c727f5f4b8c81 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:12:09.112319 22951 raft_consensus.cc:686] T 7f12d76f888b4812ac534249a705ce76 P d35b7a1a1def4fd39a7c727f5f4b8c81 [term 1 LEADER]: Becoming Leader. State: Replica: d35b7a1a1def4fd39a7c727f5f4b8c81, State: Running, Role: LEADER
I20260809 15:12:09.113003 22951 consensus_queue.cc:237] T 7f12d76f888b4812ac534249a705ce76 P d35b7a1a1def4fd39a7c727f5f4b8c81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6a0168984604c159b1ce3573320761d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39891 } } peers { permanent_uuid: "d35b7a1a1def4fd39a7c727f5f4b8c81" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 37189 } } peers { permanent_uuid: "fe558a4930c8421e9698fa907059c9fb" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38397 } }
I20260809 15:12:09.117471 22237 catalog_manager.cc:5495] T 7f12d76f888b4812ac534249a705ce76 P d35b7a1a1def4fd39a7c727f5f4b8c81 reported cstate change: term changed from 0 to 1, leader changed from <none> to d35b7a1a1def4fd39a7c727f5f4b8c81 (127.24.183.194). New cstate: current_term: 1 leader_uuid: "d35b7a1a1def4fd39a7c727f5f4b8c81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6a0168984604c159b1ce3573320761d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35b7a1a1def4fd39a7c727f5f4b8c81" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 37189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fe558a4930c8421e9698fa907059c9fb" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38397 } health_report { overall_health: UNKNOWN } } }
I20260809 15:12:09.167634 22938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:09.162555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54676 (local address 127.24.183.194:37189)
0809 15:12:09.162774 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:09.162781 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:09.162895 (+ 114us) server_negotiation.cc:407] Connection header received
0809 15:12:09.163080 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:09.163084 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:09.163174 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:09.163297 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:12:09.163647 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:09.164403 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:09.165696 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:09.166086 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:09.166172 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:09.166577 (+ 405us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:12:09.167090 (+ 513us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:12:09.167213 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:09.167293 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 15:12:09.167335 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 15:12:09.176309 22450 raft_consensus.cc:1261] T 7f12d76f888b4812ac534249a705ce76 P f6a0168984604c159b1ce3573320761d [term 1 FOLLOWER]: Refusing update from remote peer d35b7a1a1def4fd39a7c727f5f4b8c81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:12:09.176309 22852 raft_consensus.cc:1261] T 7f12d76f888b4812ac534249a705ce76 P fe558a4930c8421e9698fa907059c9fb [term 1 FOLLOWER]: Refusing update from remote peer d35b7a1a1def4fd39a7c727f5f4b8c81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:12:09.177261 22957 consensus_queue.cc:1035] T 7f12d76f888b4812ac534249a705ce76 P d35b7a1a1def4fd39a7c727f5f4b8c81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6a0168984604c159b1ce3573320761d" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:12:09.177656 22951 consensus_queue.cc:1035] T 7f12d76f888b4812ac534249a705ce76 P d35b7a1a1def4fd39a7c727f5f4b8c81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe558a4930c8421e9698fa907059c9fb" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 38397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:12:09.191202 22962 mvcc.cc:205] Tried to move back new op lower bound from 7316636996295122944 to 7316636996050788352. Current Snapshot: MvccSnapshot[applied={T|T < 7316636996295122944}]
I20260809 15:12:09.196554 22963 mvcc.cc:205] Tried to move back new op lower bound from 7316636996295122944 to 7316636996050788352. Current Snapshot: MvccSnapshot[applied={T|T < 7316636996295122944}]
I20260809 15:12:09.204209 22234 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47006:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:12:09.204543 22234 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47006:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:12:09.208422 22234 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3cd0bc4b4d13450a94a411088844752c: Sending DeleteTablet for 3 replicas of tablet 7f12d76f888b4812ac534249a705ce76
I20260809 15:12:09.209600 22619 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f12d76f888b4812ac534249a705ce76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:12:09 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:54668
I20260809 15:12:09.209591 22422 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f12d76f888b4812ac534249a705ce76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:12:09 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:59276
I20260809 15:12:09.210865 22826 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f12d76f888b4812ac534249a705ce76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:12:09 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:34074
I20260809 15:12:09.214018 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 22323
I20260809 15:12:09.215124 22972 tablet_replica.cc:330] T 7f12d76f888b4812ac534249a705ce76 P d35b7a1a1def4fd39a7c727f5f4b8c81: stopping tablet replica
I20260809 15:12:09.216078 22973 tablet_replica.cc:330] T 7f12d76f888b4812ac534249a705ce76 P f6a0168984604c159b1ce3573320761d: stopping tablet replica
I20260809 15:12:09.216466 22973 raft_consensus.cc:2229] T 7f12d76f888b4812ac534249a705ce76 P f6a0168984604c159b1ce3573320761d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:12:09.216995 22973 raft_consensus.cc:2258] T 7f12d76f888b4812ac534249a705ce76 P f6a0168984604c159b1ce3573320761d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:12:09.215545 22972 raft_consensus.cc:2229] T 7f12d76f888b4812ac534249a705ce76 P d35b7a1a1def4fd39a7c727f5f4b8c81 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:12:09.218021 22972 raft_consensus.cc:2258] T 7f12d76f888b4812ac534249a705ce76 P d35b7a1a1def4fd39a7c727f5f4b8c81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:12:09.218793 22974 tablet_replica.cc:330] T 7f12d76f888b4812ac534249a705ce76 P fe558a4930c8421e9698fa907059c9fb: stopping tablet replica
I20260809 15:12:09.219198 22974 raft_consensus.cc:2229] T 7f12d76f888b4812ac534249a705ce76 P fe558a4930c8421e9698fa907059c9fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:12:09.220820 22973 ts_tablet_manager.cc:1905] T 7f12d76f888b4812ac534249a705ce76 P f6a0168984604c159b1ce3573320761d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:12:09.221376 22972 ts_tablet_manager.cc:1905] T 7f12d76f888b4812ac534249a705ce76 P d35b7a1a1def4fd39a7c727f5f4b8c81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:12:09.227521 22974 raft_consensus.cc:2258] T 7f12d76f888b4812ac534249a705ce76 P fe558a4930c8421e9698fa907059c9fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:12:09.228966 22973 ts_tablet_manager.cc:1918] T 7f12d76f888b4812ac534249a705ce76 P f6a0168984604c159b1ce3573320761d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:12:09.229176 22973 log.cc:1196] T 7f12d76f888b4812ac534249a705ce76 P f6a0168984604c159b1ce3573320761d: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-0/wal/wals/7f12d76f888b4812ac534249a705ce76
I20260809 15:12:09.229669 22974 ts_tablet_manager.cc:1905] T 7f12d76f888b4812ac534249a705ce76 P fe558a4930c8421e9698fa907059c9fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:12:09.229899 22973 ts_tablet_manager.cc:1939] T 7f12d76f888b4812ac534249a705ce76 P f6a0168984604c159b1ce3573320761d: Deleting consensus metadata
I20260809 15:12:09.231848 22209 catalog_manager.cc:4841] TS f6a0168984604c159b1ce3573320761d (127.24.183.193:39891): tablet 7f12d76f888b4812ac534249a705ce76 (table test-table [id=538ba344d9da443587a93843df68bbbd]) successfully deleted
I20260809 15:12:09.235785 22974 ts_tablet_manager.cc:1918] T 7f12d76f888b4812ac534249a705ce76 P fe558a4930c8421e9698fa907059c9fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:12:09.235960 22974 log.cc:1196] T 7f12d76f888b4812ac534249a705ce76 P fe558a4930c8421e9698fa907059c9fb: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-2/wal/wals/7f12d76f888b4812ac534249a705ce76
I20260809 15:12:09.236495 22974 ts_tablet_manager.cc:1939] T 7f12d76f888b4812ac534249a705ce76 P fe558a4930c8421e9698fa907059c9fb: Deleting consensus metadata
I20260809 15:12:09.236522 22972 ts_tablet_manager.cc:1918] T 7f12d76f888b4812ac534249a705ce76 P d35b7a1a1def4fd39a7c727f5f4b8c81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:12:09.236786 22972 log.cc:1196] T 7f12d76f888b4812ac534249a705ce76 P d35b7a1a1def4fd39a7c727f5f4b8c81: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288021313853-25311-0/minicluster-data/ts-1/wal/wals/7f12d76f888b4812ac534249a705ce76
I20260809 15:12:09.237417 22972 ts_tablet_manager.cc:1939] T 7f12d76f888b4812ac534249a705ce76 P d35b7a1a1def4fd39a7c727f5f4b8c81: Deleting consensus metadata
I20260809 15:12:09.238266 22209 catalog_manager.cc:4841] TS fe558a4930c8421e9698fa907059c9fb (127.24.183.195:38397): tablet 7f12d76f888b4812ac534249a705ce76 (table test-table [id=538ba344d9da443587a93843df68bbbd]) successfully deleted
Aug 09 15:12:09 dist-test-slave-nj21 krb5kdc[22161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.193@KRBTEST.COM
I20260809 15:12:09.239238 22210 catalog_manager.cc:4841] TS d35b7a1a1def4fd39a7c727f5f4b8c81 (127.24.183.194:37189): tablet 7f12d76f888b4812ac534249a705ce76 (table test-table [id=538ba344d9da443587a93843df68bbbd]) successfully deleted
I20260809 15:12:09.245137 22935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:09.215920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59284 (local address 127.24.183.193:39891)
0809 15:12:09.216091 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:09.216097 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:09.228470 (+ 12373us) server_negotiation.cc:407] Connection header received
0809 15:12:09.228664 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:09.228670 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:09.228764 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:09.228873 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:09.230848 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:09.231960 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:09.233723 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:09.234368 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:09.234492 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:09.239519 (+ 5027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:09.239532 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:09.239546 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:12:09.239583 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:09.242324 (+ 2741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:09.242999 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:09.243007 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:09.243009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:09.243147 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:09.243527 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:09.243531 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:09.243533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:09.243986 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:09.244265 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:09.244568 (+ 303us) server_negotiation.cc:299] Negotiation successful
0809 15:12:09.244778 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 15:12:09.245774 22508 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:12:09.335659 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 22323
I20260809 15:12:09.364209 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 22528
Aug 09 15:12:09 dist-test-slave-nj21 krb5kdc[22161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.194@KRBTEST.COM
I20260809 15:12:09.379807 22938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:09.365004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54678 (local address 127.24.183.194:37189)
0809 15:12:09.365268 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:09.365274 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:09.365295 (+ 21us) server_negotiation.cc:407] Connection header received
0809 15:12:09.365506 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:09.365510 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:09.365576 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:09.365660 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:09.366960 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:09.367863 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:09.369161 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:09.369648 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:09.369762 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:09.374075 (+ 4313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:09.374087 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:09.374101 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:12:09.374141 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:09.377090 (+ 2949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:09.377752 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:09.377759 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:09.377762 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:09.377910 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:09.378265 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:09.378269 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:09.378271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:09.378615 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:09.378970 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:09.379202 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 15:12:09.379440 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260809 15:12:09.380493 22713 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:12:09.499310 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 22528
I20260809 15:12:09.525487 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 22732
Aug 09 15:12:09 dist-test-slave-nj21 krb5kdc[22161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.195@KRBTEST.COM
I20260809 15:12:09.540508 22940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:09.526199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34078 (local address 127.24.183.195:38397)
0809 15:12:09.526369 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:09.526374 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:09.526393 (+ 19us) server_negotiation.cc:407] Connection header received
0809 15:12:09.526538 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:09.526542 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:09.526609 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:09.526691 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:09.528117 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:09.528844 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:09.530438 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:09.530854 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:09.530950 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:09.535322 (+ 4372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:09.535348 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:09.535366 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:12:09.535402 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:09.538152 (+ 2750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:09.538734 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:09.538742 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:09.538744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:09.538876 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:09.539144 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:09.539147 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:09.539149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:09.539525 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:09.539947 (+ 422us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:09.540068 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 15:12:09.540238 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 15:12:09.541103 22908 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:12:09.629776 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 22732
I20260809 15:12:09.652436 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 22185
I20260809 15:12:09.653548 22281 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:12:09.769666 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 22185
2026-08-09T15:12:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4573 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T15:12:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:12:09Z Disabled control of system clock
I20260809 15:12:09.858457 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:38185
--webserver_interface=127.24.183.254
--webserver_port=0
--builtin_ntp_servers=127.24.183.212:38521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:38185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 15:12:10.191515 22987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:10.191835 22987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:10.191972 22987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:10.199445 22987 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:12:10.199566 22987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:10.199631 22987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:10.199729 22987 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:12:10.199793 22987 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:12:10.209244 22987 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:38521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:38185
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:38185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:10.210561 22987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:10.212420 22987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:10.223539 22994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:10.223580 22997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:10.223618 23000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:10.224637 22987 server_base.cc:1034] running on GCE node
I20260809 15:12:10.225342 22987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:10.226643 22987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:10.227845 22987 hybrid_clock.cc:648] HybridClock initialized: now 1786288330227789 us; error 33 us; skew 500 ppm
I20260809 15:12:10.228183 22987 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:12:10.230275 22987 webserver.cc:460] Webserver started at http://127.24.183.254:45995/ using document root <none> and password file <none>
I20260809 15:12:10.230918 22987 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:10.231017 22987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:10.231321 22987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:10.234184 22987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "3f88f1f9041244d9bd6844b2b722dabb"
format_stamp: "Formatted at 2026-08-09 15:12:10 on dist-test-slave-nj21"
server_key: "23da5431084fd9d0968e5d14fb1b973f"
server_key_iv: "7785b2cff4b1eb4425e133f41d3bc68c"
server_key_version: "encryptionkey@0"
I20260809 15:12:10.234974 22987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "3f88f1f9041244d9bd6844b2b722dabb"
format_stamp: "Formatted at 2026-08-09 15:12:10 on dist-test-slave-nj21"
server_key: "23da5431084fd9d0968e5d14fb1b973f"
server_key_iv: "7785b2cff4b1eb4425e133f41d3bc68c"
server_key_version: "encryptionkey@0"
I20260809 15:12:10.239899 22987 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 15:12:10.243105 23006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:10.244345 22987 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:12:10.244505 22987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "3f88f1f9041244d9bd6844b2b722dabb"
format_stamp: "Formatted at 2026-08-09 15:12:10 on dist-test-slave-nj21"
server_key: "23da5431084fd9d0968e5d14fb1b973f"
server_key_iv: "7785b2cff4b1eb4425e133f41d3bc68c"
server_key_version: "encryptionkey@0"
I20260809 15:12:10.244689 22987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:10.261194 22987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:10.261974 22987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:10.262230 22987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:10.279595 22987 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:38185
I20260809 15:12:10.279613 23079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:38185 every 8 connection(s)
I20260809 15:12:10.280910 22987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:12:10.284503 23081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:10.291258 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 22987
I20260809 15:12:10.291441 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:12:10.291934 25311 external_mini_cluster.cc:1428] Setting key 09f07e1b2265f3fabca4773ed131bd15
I20260809 15:12:10.294973 23081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f88f1f9041244d9bd6844b2b722dabb: Bootstrap starting.
I20260809 15:12:10.298395 23081 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f88f1f9041244d9bd6844b2b722dabb: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:10.299453 23081 log.cc:824] T 00000000000000000000000000000000 P 3f88f1f9041244d9bd6844b2b722dabb: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:10.302210 23081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f88f1f9041244d9bd6844b2b722dabb: No bootstrap required, opened a new log
I20260809 15:12:10.305181 23086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:10.293839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47378 (local address 127.24.183.254:38185)
0809 15:12:10.294845 (+ 1006us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:10.294855 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:10.294898 (+ 43us) server_negotiation.cc:407] Connection header received
0809 15:12:10.296723 (+ 1825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:10.296751 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:10.297239 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:10.297541 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:10.298155 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:10.299531 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:10.300917 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:10.301577 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:10.301752 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:10.302901 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:10.302912 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:10.302926 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:10.303440 (+ 514us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:10.303512 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:10.303547 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:10.303664 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:10.303937 (+ 273us) server_negotiation.cc:299] Negotiation successful
0809 15:12:10.304218 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":342,"threads_started":1}
I20260809 15:12:10.309897 23081 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f88f1f9041244d9bd6844b2b722dabb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f88f1f9041244d9bd6844b2b722dabb" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 38185 } }
I20260809 15:12:10.310230 23081 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f88f1f9041244d9bd6844b2b722dabb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:10.310321 23081 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f88f1f9041244d9bd6844b2b722dabb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f88f1f9041244d9bd6844b2b722dabb, State: Initialized, Role: FOLLOWER
I20260809 15:12:10.310840 23081 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f88f1f9041244d9bd6844b2b722dabb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f88f1f9041244d9bd6844b2b722dabb" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 38185 } }
I20260809 15:12:10.311071 23081 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f88f1f9041244d9bd6844b2b722dabb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:12:10.311203 23081 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f88f1f9041244d9bd6844b2b722dabb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:12:10.311353 23081 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f88f1f9041244d9bd6844b2b722dabb [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:10.313665 23081 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f88f1f9041244d9bd6844b2b722dabb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f88f1f9041244d9bd6844b2b722dabb" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 38185 } }
I20260809 15:12:10.314169 23081 leader_election.cc:302] T 00000000000000000000000000000000 P 3f88f1f9041244d9bd6844b2b722dabb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f88f1f9041244d9bd6844b2b722dabb; no voters:
I20260809 15:12:10.314678 23081 leader_election.cc:288] T 00000000000000000000000000000000 P 3f88f1f9041244d9bd6844b2b722dabb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:12:10.314810 23089 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f88f1f9041244d9bd6844b2b722dabb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:12:10.315501 23089 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f88f1f9041244d9bd6844b2b722dabb [term 1 LEADER]: Becoming Leader. State: Replica: 3f88f1f9041244d9bd6844b2b722dabb, State: Running, Role: LEADER
I20260809 15:12:10.316001 23089 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f88f1f9041244d9bd6844b2b722dabb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f88f1f9041244d9bd6844b2b722dabb" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 38185 } }
I20260809 15:12:10.316545 23081 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f88f1f9041244d9bd6844b2b722dabb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:12:10.319111 23091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f88f1f9041244d9bd6844b2b722dabb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f88f1f9041244d9bd6844b2b722dabb. Latest consensus state: current_term: 1 leader_uuid: "3f88f1f9041244d9bd6844b2b722dabb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f88f1f9041244d9bd6844b2b722dabb" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 38185 } } }
I20260809 15:12:10.319151 23090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f88f1f9041244d9bd6844b2b722dabb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f88f1f9041244d9bd6844b2b722dabb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f88f1f9041244d9bd6844b2b722dabb" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 38185 } } }
I20260809 15:12:10.319717 23090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f88f1f9041244d9bd6844b2b722dabb [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:10.323760 23091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f88f1f9041244d9bd6844b2b722dabb [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:10.327772 23102 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:12:10.332803 23102 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:12:10.341555 23102 catalog_manager.cc:1347] Generated new cluster ID: 461c27f4b13b4da38a094b22a6bb5a79
I20260809 15:12:10.341650 23102 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:12:10.353673 23102 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:12:10.354840 23102 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:12:10.364951 23102 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f88f1f9041244d9bd6844b2b722dabb: Generated new TSK 0
I20260809 15:12:10.365695 23102 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:12:10.370671 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.193:0
--local_ip_for_outbound_sockets=127.24.183.193
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:38185
--builtin_ntp_servers=127.24.183.212:38521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 15:12:10.705317 23114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:10.705593 23114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:10.705725 23114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:10.714092 23114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:10.714268 23114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:10.714437 23114 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.193
I20260809 15:12:10.725072 23114 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:38521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:38185
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.193
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:10.726464 23114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:10.728291 23114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:10.741122 23123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:10.741133 23124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:10.741173 23128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:10.741927 23114 server_base.cc:1034] running on GCE node
I20260809 15:12:10.742530 23114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:10.743418 23114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:10.744627 23114 hybrid_clock.cc:648] HybridClock initialized: now 1786288330744582 us; error 34 us; skew 500 ppm
I20260809 15:12:10.744944 23114 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:12:10.747067 23114 webserver.cc:460] Webserver started at http://127.24.183.193:39533/ using document root <none> and password file <none>
I20260809 15:12:10.747759 23114 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:10.748004 23114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:10.748317 23114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:10.751155 23114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-0/data/instance:
uuid: "8d656331e02c4bc893b39d91c8ba1b4c"
format_stamp: "Formatted at 2026-08-09 15:12:10 on dist-test-slave-nj21"
server_key: "c2b0c0b42a471bfa59c685c8d9c9a7e3"
server_key_iv: "98a9d5f603dc9f39118ac7a07c41c48a"
server_key_version: "encryptionkey@0"
I20260809 15:12:10.751948 23114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance:
uuid: "8d656331e02c4bc893b39d91c8ba1b4c"
format_stamp: "Formatted at 2026-08-09 15:12:10 on dist-test-slave-nj21"
server_key: "c2b0c0b42a471bfa59c685c8d9c9a7e3"
server_key_iv: "98a9d5f603dc9f39118ac7a07c41c48a"
server_key_version: "encryptionkey@0"
I20260809 15:12:10.756779 23114 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 15:12:10.760092 23141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:10.761341 23114 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 15:12:10.761487 23114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-0/wal
uuid: "8d656331e02c4bc893b39d91c8ba1b4c"
format_stamp: "Formatted at 2026-08-09 15:12:10 on dist-test-slave-nj21"
server_key: "c2b0c0b42a471bfa59c685c8d9c9a7e3"
server_key_iv: "98a9d5f603dc9f39118ac7a07c41c48a"
server_key_version: "encryptionkey@0"
I20260809 15:12:10.761680 23114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:10.783921 23114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:10.784662 23114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:10.784958 23114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:10.785890 23114 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:12:10.787285 23114 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:12:10.787375 23114 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:10.787477 23114 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:12:10.787556 23114 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:10.818136 23114 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.193:39451
I20260809 15:12:10.818208 23293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.193:39451 every 8 connection(s)
I20260809 15:12:10.819434 23114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
I20260809 15:12:10.820348 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 23114
I20260809 15:12:10.820520 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance
I20260809 15:12:10.821046 25311 external_mini_cluster.cc:1428] Setting key e89aea9e006d31d073ecafe2f3e38dc9
I20260809 15:12:10.824193 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.194:0
--local_ip_for_outbound_sockets=127.24.183.194
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:38185
--builtin_ntp_servers=127.24.183.212:38521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 15:12:10.832100 23297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:10.822175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:39609 (local address 127.24.183.254:38185)
0809 15:12:10.822663 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:10.822669 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:10.825914 (+ 3245us) server_negotiation.cc:407] Connection header received
0809 15:12:10.825968 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:10.825972 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:10.826033 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:10.826122 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:10.827196 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:10.827959 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:10.829479 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:10.829875 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:10.829981 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:10.831269 (+ 1288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:10.831274 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:10.831277 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:10.831293 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:10.831334 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:10.831346 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:10.831427 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:10.831630 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 15:12:10.831752 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":240,"threads_started":1}
I20260809 15:12:10.833039 23300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:10.822895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:38185 (local address 127.24.183.193:39609)
0809 15:12:10.823932 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:10.823969 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:10.825660 (+ 1691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:10.826367 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:10.826375 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:10.826463 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:12:10.827033 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:10.827052 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:10.828111 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:10.828117 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:10.829323 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:10.829337 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:10.830094 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:10.830102 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:10.830336 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:10.830968 (+ 632us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:12:10.830973 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:12:10.831024 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:12:10.831103 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:12:10.831432 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:12:10.831445 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:12:10.831452 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 15:12:10.831607 (+ 155us) client_negotiation.cc:240] Negotiation successful
0809 15:12:10.832010 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":304,"threads_started":1}
I20260809 15:12:10.834535 23294 heartbeater.cc:347] Connected to a master server at 127.24.183.254:38185
I20260809 15:12:10.834807 23294 heartbeater.cc:464] Registering TS with master...
I20260809 15:12:10.835512 23294 heartbeater.cc:511] Master 127.24.183.254:38185 requested a full tablet report, sending...
I20260809 15:12:10.837553 23030 ts_manager.cc:194] Registered new tserver with Master: 8d656331e02c4bc893b39d91c8ba1b4c (127.24.183.193:39451)
I20260809 15:12:10.838999 23030 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.183.193:39609
W20260809 15:12:11.162871 23302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:11.163138 23302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:11.163267 23302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:11.170806 23302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:11.170907 23302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:11.171056 23302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.194
I20260809 15:12:11.180006 23302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:38521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:38185
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.194
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:11.181296 23302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:11.183086 23302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:11.195755 23309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:11.195749 23310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:11.195873 23313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:11.196735 23302 server_base.cc:1034] running on GCE node
I20260809 15:12:11.197383 23302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:11.198299 23302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:11.199508 23302 hybrid_clock.cc:648] HybridClock initialized: now 1786288331199455 us; error 39 us; skew 500 ppm
I20260809 15:12:11.199833 23302 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:12:11.201941 23302 webserver.cc:460] Webserver started at http://127.24.183.194:34981/ using document root <none> and password file <none>
I20260809 15:12:11.202589 23302 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:11.202687 23302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:11.202984 23302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:11.205816 23302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-1/data/instance:
uuid: "e4694f1b78024f7fa5625e16dfdca107"
format_stamp: "Formatted at 2026-08-09 15:12:11 on dist-test-slave-nj21"
server_key: "d50a07f191d89fd5db50946ef613097b"
server_key_iv: "7c9c401bf7ab20df9cd008d3379a7820"
server_key_version: "encryptionkey@0"
I20260809 15:12:11.206575 23302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance:
uuid: "e4694f1b78024f7fa5625e16dfdca107"
format_stamp: "Formatted at 2026-08-09 15:12:11 on dist-test-slave-nj21"
server_key: "d50a07f191d89fd5db50946ef613097b"
server_key_iv: "7c9c401bf7ab20df9cd008d3379a7820"
server_key_version: "encryptionkey@0"
I20260809 15:12:11.211447 23302 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 15:12:11.214666 23325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:11.215919 23302 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 15:12:11.216070 23302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-1/wal
uuid: "e4694f1b78024f7fa5625e16dfdca107"
format_stamp: "Formatted at 2026-08-09 15:12:11 on dist-test-slave-nj21"
server_key: "d50a07f191d89fd5db50946ef613097b"
server_key_iv: "7c9c401bf7ab20df9cd008d3379a7820"
server_key_version: "encryptionkey@0"
I20260809 15:12:11.216256 23302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:11.231772 23302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:11.232548 23302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:11.232825 23302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:11.233703 23302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:12:11.235085 23302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:12:11.235193 23302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:11.235296 23302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:12:11.235374 23302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:11.265444 23302 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.194:44117
I20260809 15:12:11.265514 23490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.194:44117 every 8 connection(s)
I20260809 15:12:11.266829 23302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
I20260809 15:12:11.276100 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 23302
I20260809 15:12:11.276317 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance
I20260809 15:12:11.276831 25311 external_mini_cluster.cc:1428] Setting key ff202ddbbbf2b5fff17abe44dc392351
I20260809 15:12:11.279479 23297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:11.269354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:54701 (local address 127.24.183.254:38185)
0809 15:12:11.269574 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:11.269580 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:11.270998 (+ 1418us) server_negotiation.cc:407] Connection header received
0809 15:12:11.272824 (+ 1826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:11.272828 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:11.272914 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:11.272993 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:11.273976 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:11.274701 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:11.276496 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:11.277065 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:11.277176 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:11.278594 (+ 1418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:11.278599 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:11.278602 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:11.278623 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:11.278665 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:11.278682 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:11.278778 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:11.279037 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 15:12:11.279141 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 15:12:11.280104 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.195:0
--local_ip_for_outbound_sockets=127.24.183.195
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:38185
--builtin_ntp_servers=127.24.183.212:38521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 15:12:11.280798 23494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:11.269927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:38185 (local address 127.24.183.194:54701)
0809 15:12:11.270847 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:11.270884 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:11.272554 (+ 1670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:11.273184 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:11.273191 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:11.273281 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:12:11.273847 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:11.273863 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:11.274838 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:11.274842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:11.276360 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:11.276375 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:11.277306 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:11.277314 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:11.277544 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:11.278304 (+ 760us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:12:11.278307 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:12:11.278345 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:12:11.278418 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:12:11.278807 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:12:11.278816 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:12:11.278824 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 15:12:11.279007 (+ 183us) client_negotiation.cc:240] Negotiation successful
0809 15:12:11.279478 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":297,"threads_started":1}
I20260809 15:12:11.285033 23491 heartbeater.cc:347] Connected to a master server at 127.24.183.254:38185
I20260809 15:12:11.285329 23491 heartbeater.cc:464] Registering TS with master...
I20260809 15:12:11.286201 23491 heartbeater.cc:511] Master 127.24.183.254:38185 requested a full tablet report, sending...
I20260809 15:12:11.287796 23030 ts_manager.cc:194] Registered new tserver with Master: e4694f1b78024f7fa5625e16dfdca107 (127.24.183.194:44117)
I20260809 15:12:11.288970 23030 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.183.194:54701
W20260809 15:12:11.621305 23496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:11.621570 23496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:11.621698 23496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:11.628875 23496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:11.628969 23496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:11.629112 23496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.195
I20260809 15:12:11.638190 23496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:38521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:38185
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.195
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:11.639532 23496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:11.641353 23496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:11.654196 23503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:11.654140 23507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:11.654233 23505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:11.654963 23496 server_base.cc:1034] running on GCE node
I20260809 15:12:11.655525 23496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:11.656476 23496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:11.657682 23496 hybrid_clock.cc:648] HybridClock initialized: now 1786288331657626 us; error 47 us; skew 500 ppm
I20260809 15:12:11.658006 23496 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:12:11.660247 23496 webserver.cc:460] Webserver started at http://127.24.183.195:38245/ using document root <none> and password file <none>
I20260809 15:12:11.660924 23496 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:11.661027 23496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:11.661334 23496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:11.664218 23496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-2/data/instance:
uuid: "5384b2937fea4de7bb861da993f3d3c4"
format_stamp: "Formatted at 2026-08-09 15:12:11 on dist-test-slave-nj21"
server_key: "4082c10662f1c18261a94a8bb0961623"
server_key_iv: "355acaa70645e8fc3c7ab9aaec7c68c9"
server_key_version: "encryptionkey@0"
I20260809 15:12:11.665030 23496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance:
uuid: "5384b2937fea4de7bb861da993f3d3c4"
format_stamp: "Formatted at 2026-08-09 15:12:11 on dist-test-slave-nj21"
server_key: "4082c10662f1c18261a94a8bb0961623"
server_key_iv: "355acaa70645e8fc3c7ab9aaec7c68c9"
server_key_version: "encryptionkey@0"
I20260809 15:12:11.670095 23496 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 15:12:11.673461 23513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:11.674817 23496 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 15:12:11.674980 23496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-2/wal
uuid: "5384b2937fea4de7bb861da993f3d3c4"
format_stamp: "Formatted at 2026-08-09 15:12:11 on dist-test-slave-nj21"
server_key: "4082c10662f1c18261a94a8bb0961623"
server_key_iv: "355acaa70645e8fc3c7ab9aaec7c68c9"
server_key_version: "encryptionkey@0"
I20260809 15:12:11.675203 23496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:11.704025 23496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:11.704905 23496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:11.705188 23496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:11.706182 23496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:12:11.709337 23496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:12:11.709455 23496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:11.709566 23496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:12:11.709636 23496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:11.741127 23496 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.195:34471
I20260809 15:12:11.741175 23687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.195:34471 every 8 connection(s)
I20260809 15:12:11.742419 23496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
I20260809 15:12:11.751343 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 23496
I20260809 15:12:11.751513 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance
I20260809 15:12:11.751982 25311 external_mini_cluster.cc:1428] Setting key 6aa8eb2c48dbeba84b8360a19abc3c09
I20260809 15:12:11.754948 23297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:11.744954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:38921 (local address 127.24.183.254:38185)
0809 15:12:11.745178 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:11.745185 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:11.746533 (+ 1348us) server_negotiation.cc:407] Connection header received
0809 15:12:11.748463 (+ 1930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:11.748467 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:11.748538 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:11.748618 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:11.749716 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:11.750477 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:11.752135 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:11.752701 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:11.752813 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:11.754166 (+ 1353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:11.754171 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:11.754174 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:11.754192 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:11.754223 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:11.754235 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:11.754309 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:11.754526 (+ 217us) server_negotiation.cc:299] Negotiation successful
0809 15:12:11.754627 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 15:12:11.755944 23691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:11.745494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:38185 (local address 127.24.183.195:38921)
0809 15:12:11.746382 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:11.746418 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:11.748203 (+ 1785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:11.748792 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:11.748800 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:11.748892 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:12:11.749567 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:11.749594 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:11.750591 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:11.750596 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:11.751999 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:11.752011 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:11.752926 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:11.752934 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:11.753202 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:11.753877 (+ 675us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:12:11.753880 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:12:11.753929 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:12:11.753998 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:12:11.754319 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:12:11.754329 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:12:11.754337 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 15:12:11.754500 (+ 163us) client_negotiation.cc:240] Negotiation successful
0809 15:12:11.754860 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":289,"threads_started":1}
I20260809 15:12:11.757035 23688 heartbeater.cc:347] Connected to a master server at 127.24.183.254:38185
I20260809 15:12:11.757298 23688 heartbeater.cc:464] Registering TS with master...
I20260809 15:12:11.757995 23688 heartbeater.cc:511] Master 127.24.183.254:38185 requested a full tablet report, sending...
I20260809 15:12:11.759270 23030 ts_manager.cc:194] Registered new tserver with Master: 5384b2937fea4de7bb861da993f3d3c4 (127.24.183.195:34471)
I20260809 15:12:11.760071 23030 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.183.195:38921
I20260809 15:12:11.766696 25311 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:12:11.776084 23297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:11.770780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47388 (local address 127.24.183.254:38185)
0809 15:12:11.770997 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:11.771003 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:11.771354 (+ 351us) server_negotiation.cc:407] Connection header received
0809 15:12:11.771477 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:11.771481 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:11.771548 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:11.771619 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:11.772156 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:11.772827 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:11.774073 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:11.774432 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:11.774513 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:11.775433 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:11.775438 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:11.775441 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:11.775457 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:11.775488 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:11.775499 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:11.775602 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:11.775735 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 15:12:11.775776 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 15:12:11.779651 23030 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47388:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:12:11.786327 23030 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:12:11.818720 23704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:11.797631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:44117 (local address 127.0.0.1:36910)
0809 15:12:11.809760 (+ 12129us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:11.809773 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:11.809881 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:11.810767 (+ 886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:11.810774 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:11.810802 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:12:11.811536 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:11.811562 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:11.813638 (+ 2076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:11.813644 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:11.815819 (+ 2175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:11.815846 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:11.817842 (+ 1996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:11.817849 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:11.818036 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:11.818088 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 15:12:11.818198 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 15:12:11.818397 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12002,"mutex_wait_us":228,"thread_start_us":11971,"threads_started":1}
I20260809 15:12:11.819402 23705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:11.797696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60214 (local address 127.24.183.195:34471)
0809 15:12:11.810449 (+ 12753us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:11.810457 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:11.810487 (+ 30us) server_negotiation.cc:407] Connection header received
0809 15:12:11.810578 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:11.810585 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:11.810807 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:11.810983 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:12:11.811965 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:11.813491 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:11.816637 (+ 3146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:11.817909 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:11.818082 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:11.818270 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:11.818845 (+ 575us) server_negotiation.cc:299] Negotiation successful
0809 15:12:11.819041 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12567,"thread_start_us":12415,"threads_started":1}
I20260809 15:12:11.820039 23706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:11.797874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36910 (local address 127.24.183.194:44117)
0809 15:12:11.808713 (+ 10839us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:11.808734 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:11.809877 (+ 1143us) server_negotiation.cc:407] Connection header received
0809 15:12:11.810032 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:11.810041 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:11.810291 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:11.810490 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:12:11.811789 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:11.813471 (+ 1682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:11.815996 (+ 2525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:11.817323 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:11.817484 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:11.817718 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:11.819177 (+ 1459us) server_negotiation.cc:299] Negotiation successful
0809 15:12:11.819354 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10690,"thread_start_us":10576,"threads_started":1}
I20260809 15:12:11.819233 23703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:11.797406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:34471 (local address 127.0.0.1:60214)
0809 15:12:11.809157 (+ 11751us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:11.809205 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:11.809368 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:11.811008 (+ 1640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:11.811013 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:11.811044 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:12:11.811707 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:11.811722 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:11.813638 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:11.813644 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:11.816377 (+ 2733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:11.816399 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:11.818416 (+ 2017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:11.818423 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:11.818610 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:11.818642 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 15:12:11.818757 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 15:12:11.818884 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11527,"mutex_wait_us":237,"thread_start_us":11373,"threads_started":1}
I20260809 15:12:11.824414 23708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:11.810739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:39451 (local address 127.0.0.1:56602)
0809 15:12:11.811714 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:11.811729 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:11.811846 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:11.812330 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:11.812334 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:11.812357 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:12:11.812683 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:11.812691 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:11.814065 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:11.814074 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:11.816384 (+ 2310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:11.816394 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:11.823720 (+ 7326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:11.823727 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:11.823853 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:11.823875 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 15:12:11.823963 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 15:12:11.824088 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":818,"mutex_wait_us":6,"thread_start_us":630,"threads_started":1}
I20260809 15:12:11.824535 23707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:11.810207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56602 (local address 127.24.183.193:39451)
0809 15:12:11.810697 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:11.810705 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:11.811811 (+ 1106us) server_negotiation.cc:407] Connection header received
0809 15:12:11.811986 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:11.811994 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:11.812169 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:11.812318 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:12:11.812825 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:11.813889 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:11.816584 (+ 2695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:11.818056 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:11.818222 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:11.818411 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:11.824026 (+ 5615us) server_negotiation.cc:299] Negotiation successful
0809 15:12:11.824193 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":252,"threads_started":1}
I20260809 15:12:11.824429 23394 tablet_service.cc:1467] Processing CreateTablet for tablet 93f942245aeb4775a827ea7e51c3aac8 (DEFAULT_TABLE table=test-table [id=d4be90e84eba4c678235311b6ee8113c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:12:11.824429 23579 tablet_service.cc:1467] Processing CreateTablet for tablet 93f942245aeb4775a827ea7e51c3aac8 (DEFAULT_TABLE table=test-table [id=d4be90e84eba4c678235311b6ee8113c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:12:11.825881 23394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93f942245aeb4775a827ea7e51c3aac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:11.825887 23579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93f942245aeb4775a827ea7e51c3aac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:11.827600 23202 tablet_service.cc:1467] Processing CreateTablet for tablet 93f942245aeb4775a827ea7e51c3aac8 (DEFAULT_TABLE table=test-table [id=d4be90e84eba4c678235311b6ee8113c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:12:11.828742 23202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93f942245aeb4775a827ea7e51c3aac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:11.839179 23709 tablet_bootstrap.cc:492] T 93f942245aeb4775a827ea7e51c3aac8 P 8d656331e02c4bc893b39d91c8ba1b4c: Bootstrap starting.
I20260809 15:12:11.841138 23711 tablet_bootstrap.cc:492] T 93f942245aeb4775a827ea7e51c3aac8 P e4694f1b78024f7fa5625e16dfdca107: Bootstrap starting.
I20260809 15:12:11.842285 23713 tablet_bootstrap.cc:492] T 93f942245aeb4775a827ea7e51c3aac8 P 5384b2937fea4de7bb861da993f3d3c4: Bootstrap starting.
I20260809 15:12:11.842280 23294 heartbeater.cc:503] Master 127.24.183.254:38185 was elected leader, sending a full tablet report...
I20260809 15:12:11.843073 23709 tablet_bootstrap.cc:654] T 93f942245aeb4775a827ea7e51c3aac8 P 8d656331e02c4bc893b39d91c8ba1b4c: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:11.845798 23713 tablet_bootstrap.cc:654] T 93f942245aeb4775a827ea7e51c3aac8 P 5384b2937fea4de7bb861da993f3d3c4: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:11.845858 23711 tablet_bootstrap.cc:654] T 93f942245aeb4775a827ea7e51c3aac8 P e4694f1b78024f7fa5625e16dfdca107: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:11.846978 23711 log.cc:824] T 93f942245aeb4775a827ea7e51c3aac8 P e4694f1b78024f7fa5625e16dfdca107: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:11.847018 23713 log.cc:824] T 93f942245aeb4775a827ea7e51c3aac8 P 5384b2937fea4de7bb861da993f3d3c4: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:11.847642 23709 log.cc:824] T 93f942245aeb4775a827ea7e51c3aac8 P 8d656331e02c4bc893b39d91c8ba1b4c: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:11.850344 23711 tablet_bootstrap.cc:492] T 93f942245aeb4775a827ea7e51c3aac8 P e4694f1b78024f7fa5625e16dfdca107: No bootstrap required, opened a new log
I20260809 15:12:11.850443 23713 tablet_bootstrap.cc:492] T 93f942245aeb4775a827ea7e51c3aac8 P 5384b2937fea4de7bb861da993f3d3c4: No bootstrap required, opened a new log
I20260809 15:12:11.850641 23711 ts_tablet_manager.cc:1397] T 93f942245aeb4775a827ea7e51c3aac8 P e4694f1b78024f7fa5625e16dfdca107: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.005s
I20260809 15:12:11.850665 23713 ts_tablet_manager.cc:1397] T 93f942245aeb4775a827ea7e51c3aac8 P 5384b2937fea4de7bb861da993f3d3c4: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260809 15:12:11.853808 23709 tablet_bootstrap.cc:492] T 93f942245aeb4775a827ea7e51c3aac8 P 8d656331e02c4bc893b39d91c8ba1b4c: No bootstrap required, opened a new log
I20260809 15:12:11.854044 23709 ts_tablet_manager.cc:1397] T 93f942245aeb4775a827ea7e51c3aac8 P 8d656331e02c4bc893b39d91c8ba1b4c: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.000s
I20260809 15:12:11.860989 23711 raft_consensus.cc:348] T 93f942245aeb4775a827ea7e51c3aac8 P e4694f1b78024f7fa5625e16dfdca107 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5384b2937fea4de7bb861da993f3d3c4" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 34471 } } peers { permanent_uuid: "e4694f1b78024f7fa5625e16dfdca107" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 44117 } } peers { permanent_uuid: "8d656331e02c4bc893b39d91c8ba1b4c" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39451 } }
I20260809 15:12:11.860989 23713 raft_consensus.cc:348] T 93f942245aeb4775a827ea7e51c3aac8 P 5384b2937fea4de7bb861da993f3d3c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5384b2937fea4de7bb861da993f3d3c4" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 34471 } } peers { permanent_uuid: "e4694f1b78024f7fa5625e16dfdca107" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 44117 } } peers { permanent_uuid: "8d656331e02c4bc893b39d91c8ba1b4c" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39451 } }
I20260809 15:12:11.861438 23713 raft_consensus.cc:374] T 93f942245aeb4775a827ea7e51c3aac8 P 5384b2937fea4de7bb861da993f3d3c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:11.861433 23711 raft_consensus.cc:374] T 93f942245aeb4775a827ea7e51c3aac8 P e4694f1b78024f7fa5625e16dfdca107 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:11.861544 23711 raft_consensus.cc:729] T 93f942245aeb4775a827ea7e51c3aac8 P e4694f1b78024f7fa5625e16dfdca107 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4694f1b78024f7fa5625e16dfdca107, State: Initialized, Role: FOLLOWER
I20260809 15:12:11.861544 23713 raft_consensus.cc:729] T 93f942245aeb4775a827ea7e51c3aac8 P 5384b2937fea4de7bb861da993f3d3c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5384b2937fea4de7bb861da993f3d3c4, State: Initialized, Role: FOLLOWER
I20260809 15:12:11.861588 23709 raft_consensus.cc:348] T 93f942245aeb4775a827ea7e51c3aac8 P 8d656331e02c4bc893b39d91c8ba1b4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5384b2937fea4de7bb861da993f3d3c4" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 34471 } } peers { permanent_uuid: "e4694f1b78024f7fa5625e16dfdca107" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 44117 } } peers { permanent_uuid: "8d656331e02c4bc893b39d91c8ba1b4c" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39451 } }
I20260809 15:12:11.862016 23709 raft_consensus.cc:374] T 93f942245aeb4775a827ea7e51c3aac8 P 8d656331e02c4bc893b39d91c8ba1b4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:11.862124 23709 raft_consensus.cc:729] T 93f942245aeb4775a827ea7e51c3aac8 P 8d656331e02c4bc893b39d91c8ba1b4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d656331e02c4bc893b39d91c8ba1b4c, State: Initialized, Role: FOLLOWER
I20260809 15:12:11.862155 23713 consensus_queue.cc:260] T 93f942245aeb4775a827ea7e51c3aac8 P 5384b2937fea4de7bb861da993f3d3c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5384b2937fea4de7bb861da993f3d3c4" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 34471 } } peers { permanent_uuid: "e4694f1b78024f7fa5625e16dfdca107" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 44117 } } peers { permanent_uuid: "8d656331e02c4bc893b39d91c8ba1b4c" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39451 } }
I20260809 15:12:11.862155 23711 consensus_queue.cc:260] T 93f942245aeb4775a827ea7e51c3aac8 P e4694f1b78024f7fa5625e16dfdca107 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5384b2937fea4de7bb861da993f3d3c4" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 34471 } } peers { permanent_uuid: "e4694f1b78024f7fa5625e16dfdca107" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 44117 } } peers { permanent_uuid: "8d656331e02c4bc893b39d91c8ba1b4c" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39451 } }
I20260809 15:12:11.862663 23709 consensus_queue.cc:260] T 93f942245aeb4775a827ea7e51c3aac8 P 8d656331e02c4bc893b39d91c8ba1b4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5384b2937fea4de7bb861da993f3d3c4" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 34471 } } peers { permanent_uuid: "e4694f1b78024f7fa5625e16dfdca107" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 44117 } } peers { permanent_uuid: "8d656331e02c4bc893b39d91c8ba1b4c" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39451 } }
I20260809 15:12:11.863798 23491 heartbeater.cc:503] Master 127.24.183.254:38185 was elected leader, sending a full tablet report...
I20260809 15:12:11.863953 23713 ts_tablet_manager.cc:1428] T 93f942245aeb4775a827ea7e51c3aac8 P 5384b2937fea4de7bb861da993f3d3c4: Time spent starting tablet: real 0.013s user 0.013s sys 0.002s
I20260809 15:12:11.864341 23711 ts_tablet_manager.cc:1428] T 93f942245aeb4775a827ea7e51c3aac8 P e4694f1b78024f7fa5625e16dfdca107: Time spent starting tablet: real 0.014s user 0.012s sys 0.003s
I20260809 15:12:11.864476 23688 heartbeater.cc:503] Master 127.24.183.254:38185 was elected leader, sending a full tablet report...
I20260809 15:12:11.866930 23709 ts_tablet_manager.cc:1428] T 93f942245aeb4775a827ea7e51c3aac8 P 8d656331e02c4bc893b39d91c8ba1b4c: Time spent starting tablet: real 0.013s user 0.010s sys 0.003s
I20260809 15:12:11.887614 23722 raft_consensus.cc:482] T 93f942245aeb4775a827ea7e51c3aac8 P 8d656331e02c4bc893b39d91c8ba1b4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:12:11.888027 23722 raft_consensus.cc:504] T 93f942245aeb4775a827ea7e51c3aac8 P 8d656331e02c4bc893b39d91c8ba1b4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5384b2937fea4de7bb861da993f3d3c4" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 34471 } } peers { permanent_uuid: "e4694f1b78024f7fa5625e16dfdca107" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 44117 } } peers { permanent_uuid: "8d656331e02c4bc893b39d91c8ba1b4c" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39451 } }
I20260809 15:12:11.889930 23722 leader_election.cc:288] T 93f942245aeb4775a827ea7e51c3aac8 P 8d656331e02c4bc893b39d91c8ba1b4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5384b2937fea4de7bb861da993f3d3c4 (127.24.183.195:34471), e4694f1b78024f7fa5625e16dfdca107 (127.24.183.194:44117)
I20260809 15:12:11.897169 23705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:11.890355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:45723 (local address 127.24.183.195:34471)
0809 15:12:11.890613 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:11.890619 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:11.891341 (+ 722us) server_negotiation.cc:407] Connection header received
0809 15:12:11.891504 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:11.891509 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:11.891593 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:11.891720 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:12:11.892360 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:11.893163 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:11.895362 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:11.896314 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:11.896454 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:11.896526 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:11.896775 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 15:12:11.896906 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260809 15:12:11.897233 23724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:11.890372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:34471 (local address 127.24.183.193:45723)
0809 15:12:11.891228 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:11.891243 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:11.891356 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:11.891728 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:11.891732 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:11.891757 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:12:11.892196 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:11.892210 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:11.893309 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:11.893314 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:11.895164 (+ 1850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:11.895176 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:11.896493 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:11.896499 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:11.896628 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:11.896657 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 15:12:11.896736 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 15:12:11.896856 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"mutex_wait_us":118,"thread_start_us":528,"threads_started":1}
I20260809 15:12:11.897423 23706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:11.890374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:47019 (local address 127.24.183.194:44117)
0809 15:12:11.890588 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:11.890594 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:11.891498 (+ 904us) server_negotiation.cc:407] Connection header received
0809 15:12:11.891650 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:11.891655 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:11.891772 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:11.891868 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:12:11.892566 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:11.893236 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:11.895445 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:11.896355 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:11.896461 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:11.896514 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:11.896987 (+ 473us) server_negotiation.cc:299] Negotiation successful
0809 15:12:11.897124 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 15:12:11.897501 23725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:11.890413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:44117 (local address 127.24.183.193:47019)
0809 15:12:11.891379 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:11.891398 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:11.891540 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:11.891909 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:11.891913 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:11.891938 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:12:11.892375 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:11.892389 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:11.893387 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:11.893392 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:11.895171 (+ 1779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:11.895186 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:11.896668 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:11.896673 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:11.896809 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:11.896834 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 15:12:11.896938 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 15:12:11.897075 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":834,"mutex_wait_us":44,"thread_start_us":492,"threads_started":1}
I20260809 15:12:11.898339 23615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93f942245aeb4775a827ea7e51c3aac8" candidate_uuid: "8d656331e02c4bc893b39d91c8ba1b4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5384b2937fea4de7bb861da993f3d3c4" is_pre_election: true
I20260809 15:12:11.898486 23425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93f942245aeb4775a827ea7e51c3aac8" candidate_uuid: "8d656331e02c4bc893b39d91c8ba1b4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4694f1b78024f7fa5625e16dfdca107" is_pre_election: true
I20260809 15:12:11.898901 23615 raft_consensus.cc:2454] T 93f942245aeb4775a827ea7e51c3aac8 P 5384b2937fea4de7bb861da993f3d3c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d656331e02c4bc893b39d91c8ba1b4c in term 0.
I20260809 15:12:11.898968 23425 raft_consensus.cc:2454] T 93f942245aeb4775a827ea7e51c3aac8 P e4694f1b78024f7fa5625e16dfdca107 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d656331e02c4bc893b39d91c8ba1b4c in term 0.
I20260809 15:12:11.899556 23149 leader_election.cc:302] T 93f942245aeb4775a827ea7e51c3aac8 P 8d656331e02c4bc893b39d91c8ba1b4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d656331e02c4bc893b39d91c8ba1b4c, e4694f1b78024f7fa5625e16dfdca107; no voters:
I20260809 15:12:11.899940 23722 raft_consensus.cc:2781] T 93f942245aeb4775a827ea7e51c3aac8 P 8d656331e02c4bc893b39d91c8ba1b4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:12:11.900097 23722 raft_consensus.cc:482] T 93f942245aeb4775a827ea7e51c3aac8 P 8d656331e02c4bc893b39d91c8ba1b4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:12:11.900203 23722 raft_consensus.cc:3037] T 93f942245aeb4775a827ea7e51c3aac8 P 8d656331e02c4bc893b39d91c8ba1b4c [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:11.902552 23722 raft_consensus.cc:504] T 93f942245aeb4775a827ea7e51c3aac8 P 8d656331e02c4bc893b39d91c8ba1b4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5384b2937fea4de7bb861da993f3d3c4" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 34471 } } peers { permanent_uuid: "e4694f1b78024f7fa5625e16dfdca107" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 44117 } } peers { permanent_uuid: "8d656331e02c4bc893b39d91c8ba1b4c" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39451 } }
I20260809 15:12:11.903271 23722 leader_election.cc:288] T 93f942245aeb4775a827ea7e51c3aac8 P 8d656331e02c4bc893b39d91c8ba1b4c [CANDIDATE]: Term 1 election: Requested vote from peers 5384b2937fea4de7bb861da993f3d3c4 (127.24.183.195:34471), e4694f1b78024f7fa5625e16dfdca107 (127.24.183.194:44117)
I20260809 15:12:11.903594 23615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93f942245aeb4775a827ea7e51c3aac8" candidate_uuid: "8d656331e02c4bc893b39d91c8ba1b4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5384b2937fea4de7bb861da993f3d3c4"
I20260809 15:12:11.903829 23615 raft_consensus.cc:3037] T 93f942245aeb4775a827ea7e51c3aac8 P 5384b2937fea4de7bb861da993f3d3c4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:11.903800 23425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93f942245aeb4775a827ea7e51c3aac8" candidate_uuid: "8d656331e02c4bc893b39d91c8ba1b4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4694f1b78024f7fa5625e16dfdca107"
I20260809 15:12:11.904004 23425 raft_consensus.cc:3037] T 93f942245aeb4775a827ea7e51c3aac8 P e4694f1b78024f7fa5625e16dfdca107 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:11.907063 23615 raft_consensus.cc:2454] T 93f942245aeb4775a827ea7e51c3aac8 P 5384b2937fea4de7bb861da993f3d3c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d656331e02c4bc893b39d91c8ba1b4c in term 1.
I20260809 15:12:11.907063 23425 raft_consensus.cc:2454] T 93f942245aeb4775a827ea7e51c3aac8 P e4694f1b78024f7fa5625e16dfdca107 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d656331e02c4bc893b39d91c8ba1b4c in term 1.
I20260809 15:12:11.907461 23149 leader_election.cc:302] T 93f942245aeb4775a827ea7e51c3aac8 P 8d656331e02c4bc893b39d91c8ba1b4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d656331e02c4bc893b39d91c8ba1b4c, e4694f1b78024f7fa5625e16dfdca107; no voters:
I20260809 15:12:11.907831 23722 raft_consensus.cc:2781] T 93f942245aeb4775a827ea7e51c3aac8 P 8d656331e02c4bc893b39d91c8ba1b4c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:12:11.908285 23722 raft_consensus.cc:686] T 93f942245aeb4775a827ea7e51c3aac8 P 8d656331e02c4bc893b39d91c8ba1b4c [term 1 LEADER]: Becoming Leader. State: Replica: 8d656331e02c4bc893b39d91c8ba1b4c, State: Running, Role: LEADER
I20260809 15:12:11.908769 23722 consensus_queue.cc:237] T 93f942245aeb4775a827ea7e51c3aac8 P 8d656331e02c4bc893b39d91c8ba1b4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5384b2937fea4de7bb861da993f3d3c4" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 34471 } } peers { permanent_uuid: "e4694f1b78024f7fa5625e16dfdca107" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 44117 } } peers { permanent_uuid: "8d656331e02c4bc893b39d91c8ba1b4c" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39451 } }
I20260809 15:12:11.913548 23029 catalog_manager.cc:5495] T 93f942245aeb4775a827ea7e51c3aac8 P 8d656331e02c4bc893b39d91c8ba1b4c reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d656331e02c4bc893b39d91c8ba1b4c (127.24.183.193). New cstate: current_term: 1 leader_uuid: "8d656331e02c4bc893b39d91c8ba1b4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5384b2937fea4de7bb861da993f3d3c4" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 34471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4694f1b78024f7fa5625e16dfdca107" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 44117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d656331e02c4bc893b39d91c8ba1b4c" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 39451 } health_report { overall_health: HEALTHY } } }
I20260809 15:12:11.931412 23707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:11.926434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56612 (local address 127.24.183.193:39451)
0809 15:12:11.926590 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:11.926594 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:11.926776 (+ 182us) server_negotiation.cc:407] Connection header received
0809 15:12:11.926954 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:11.926959 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:11.927038 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:11.927157 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:12:11.927533 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:11.928246 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:11.929534 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:11.929942 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:11.930026 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:11.930467 (+ 441us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:12:11.930944 (+ 477us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:12:11.931062 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:11.931160 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 15:12:11.931194 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 15:12:11.939592 23615 raft_consensus.cc:1261] T 93f942245aeb4775a827ea7e51c3aac8 P 5384b2937fea4de7bb861da993f3d3c4 [term 1 FOLLOWER]: Refusing update from remote peer 8d656331e02c4bc893b39d91c8ba1b4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:12:11.939600 23425 raft_consensus.cc:1261] T 93f942245aeb4775a827ea7e51c3aac8 P e4694f1b78024f7fa5625e16dfdca107 [term 1 FOLLOWER]: Refusing update from remote peer 8d656331e02c4bc893b39d91c8ba1b4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:12:11.940629 23726 consensus_queue.cc:1035] T 93f942245aeb4775a827ea7e51c3aac8 P 8d656331e02c4bc893b39d91c8ba1b4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4694f1b78024f7fa5625e16dfdca107" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 44117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:12:11.940979 23722 consensus_queue.cc:1035] T 93f942245aeb4775a827ea7e51c3aac8 P 8d656331e02c4bc893b39d91c8ba1b4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5384b2937fea4de7bb861da993f3d3c4" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 34471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:12:11.951882 23729 mvcc.cc:205] Tried to move back new op lower bound from 7316637007613992960 to 7316637007502225408. Current Snapshot: MvccSnapshot[applied={T|T < 7316637007613992960}]
I20260809 15:12:11.964814 23029 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:12:11.965082 23029 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:12:11.968456 23029 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3f88f1f9041244d9bd6844b2b722dabb: Sending DeleteTablet for 3 replicas of tablet 93f942245aeb4775a827ea7e51c3aac8
I20260809 15:12:11.969372 23579 tablet_service.cc:1514] Processing DeleteTablet for tablet 93f942245aeb4775a827ea7e51c3aac8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:12:11 UTC) from {username='slave'} at 127.0.0.1:60214
I20260809 15:12:11.969732 23202 tablet_service.cc:1514] Processing DeleteTablet for tablet 93f942245aeb4775a827ea7e51c3aac8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:12:11 UTC) from {username='slave'} at 127.0.0.1:56602
I20260809 15:12:11.969775 23394 tablet_service.cc:1514] Processing DeleteTablet for tablet 93f942245aeb4775a827ea7e51c3aac8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:12:11 UTC) from {username='slave'} at 127.0.0.1:36910
I20260809 15:12:11.971839 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 23114
I20260809 15:12:11.980279 23744 tablet_replica.cc:330] T 93f942245aeb4775a827ea7e51c3aac8 P 8d656331e02c4bc893b39d91c8ba1b4c: stopping tablet replica
I20260809 15:12:11.980814 23744 raft_consensus.cc:2229] T 93f942245aeb4775a827ea7e51c3aac8 P 8d656331e02c4bc893b39d91c8ba1b4c [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:12:11.981453 23743 tablet_replica.cc:330] T 93f942245aeb4775a827ea7e51c3aac8 P e4694f1b78024f7fa5625e16dfdca107: stopping tablet replica
I20260809 15:12:11.982144 23744 raft_consensus.cc:2258] T 93f942245aeb4775a827ea7e51c3aac8 P 8d656331e02c4bc893b39d91c8ba1b4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:12:11.982673 23743 raft_consensus.cc:2229] T 93f942245aeb4775a827ea7e51c3aac8 P e4694f1b78024f7fa5625e16dfdca107 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:12:11.983485 23743 raft_consensus.cc:2258] T 93f942245aeb4775a827ea7e51c3aac8 P e4694f1b78024f7fa5625e16dfdca107 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:12:11.985668 23744 ts_tablet_manager.cc:1905] T 93f942245aeb4775a827ea7e51c3aac8 P 8d656331e02c4bc893b39d91c8ba1b4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:12:11.986550 23742 tablet_replica.cc:330] T 93f942245aeb4775a827ea7e51c3aac8 P 5384b2937fea4de7bb861da993f3d3c4: stopping tablet replica
I20260809 15:12:11.986939 23742 raft_consensus.cc:2229] T 93f942245aeb4775a827ea7e51c3aac8 P 5384b2937fea4de7bb861da993f3d3c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:12:11.987560 23742 raft_consensus.cc:2258] T 93f942245aeb4775a827ea7e51c3aac8 P 5384b2937fea4de7bb861da993f3d3c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:12:11.988957 23743 ts_tablet_manager.cc:1905] T 93f942245aeb4775a827ea7e51c3aac8 P e4694f1b78024f7fa5625e16dfdca107: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:12:11.991468 23742 ts_tablet_manager.cc:1905] T 93f942245aeb4775a827ea7e51c3aac8 P 5384b2937fea4de7bb861da993f3d3c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:12:11.995689 23744 ts_tablet_manager.cc:1918] T 93f942245aeb4775a827ea7e51c3aac8 P 8d656331e02c4bc893b39d91c8ba1b4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:12:11.995920 23744 log.cc:1196] T 93f942245aeb4775a827ea7e51c3aac8 P 8d656331e02c4bc893b39d91c8ba1b4c: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-0/wal/wals/93f942245aeb4775a827ea7e51c3aac8
I20260809 15:12:11.996390 23743 ts_tablet_manager.cc:1918] T 93f942245aeb4775a827ea7e51c3aac8 P e4694f1b78024f7fa5625e16dfdca107: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:12:11.996560 23744 ts_tablet_manager.cc:1939] T 93f942245aeb4775a827ea7e51c3aac8 P 8d656331e02c4bc893b39d91c8ba1b4c: Deleting consensus metadata
I20260809 15:12:11.996626 23743 log.cc:1196] T 93f942245aeb4775a827ea7e51c3aac8 P e4694f1b78024f7fa5625e16dfdca107: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-1/wal/wals/93f942245aeb4775a827ea7e51c3aac8
I20260809 15:12:11.997213 23743 ts_tablet_manager.cc:1939] T 93f942245aeb4775a827ea7e51c3aac8 P e4694f1b78024f7fa5625e16dfdca107: Deleting consensus metadata
I20260809 15:12:11.998278 23742 ts_tablet_manager.cc:1918] T 93f942245aeb4775a827ea7e51c3aac8 P 5384b2937fea4de7bb861da993f3d3c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:12:11.998486 23742 log.cc:1196] T 93f942245aeb4775a827ea7e51c3aac8 P 5384b2937fea4de7bb861da993f3d3c4: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288021313853-25311-0/minicluster-data/ts-2/wal/wals/93f942245aeb4775a827ea7e51c3aac8
I20260809 15:12:11.998517 23012 catalog_manager.cc:4841] TS 8d656331e02c4bc893b39d91c8ba1b4c (127.24.183.193:39451): tablet 93f942245aeb4775a827ea7e51c3aac8 (table test-table [id=d4be90e84eba4c678235311b6ee8113c]) successfully deleted
I20260809 15:12:11.999025 23742 ts_tablet_manager.cc:1939] T 93f942245aeb4775a827ea7e51c3aac8 P 5384b2937fea4de7bb861da993f3d3c4: Deleting consensus metadata
I20260809 15:12:11.999045 23012 catalog_manager.cc:4841] TS e4694f1b78024f7fa5625e16dfdca107 (127.24.183.194:44117): tablet 93f942245aeb4775a827ea7e51c3aac8 (table test-table [id=d4be90e84eba4c678235311b6ee8113c]) successfully deleted
I20260809 15:12:12.000624 23011 catalog_manager.cc:4841] TS 5384b2937fea4de7bb861da993f3d3c4 (127.24.183.195:34471): tablet 93f942245aeb4775a827ea7e51c3aac8 (table test-table [id=d4be90e84eba4c678235311b6ee8113c]) successfully deleted
I20260809 15:12:12.003096 23707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:11.973295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56614 (local address 127.24.183.193:39451)
0809 15:12:11.973504 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:11.973511 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:11.987033 (+ 13522us) server_negotiation.cc:407] Connection header received
0809 15:12:11.987709 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:11.987715 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:11.987819 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:11.987935 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:11.989358 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:11.990632 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:11.999828 (+ 9196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:12.000503 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:12.000628 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:12.001723 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:12.001730 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:12.001744 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:12.002332 (+ 588us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:12.002401 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:12.002422 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:12.002514 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:12.002665 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 15:12:12.002767 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 15:12:12.003854 23285 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:12:12.102788 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 23114
I20260809 15:12:12.127959 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 23302
I20260809 15:12:12.138151 23706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:12.128789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36922 (local address 127.24.183.194:44117)
0809 15:12:12.128981 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:12.128986 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:12.129006 (+ 20us) server_negotiation.cc:407] Connection header received
0809 15:12:12.129068 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:12.129071 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:12.129155 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:12.129251 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:12.129619 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:12.130404 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:12.131631 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:12.132220 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:12.132333 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:12.136847 (+ 4514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:12.136857 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:12.136874 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:12.137470 (+ 596us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:12.137524 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:12.137548 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:12.137644 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:12.137720 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 15:12:12.137819 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 15:12:12.138863 23484 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:12:12.226558 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 23302
I20260809 15:12:12.247922 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 23496
I20260809 15:12:12.254402 23705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:12.248673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60222 (local address 127.24.183.195:34471)
0809 15:12:12.248860 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:12.248865 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:12.248916 (+ 51us) server_negotiation.cc:407] Connection header received
0809 15:12:12.249066 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:12.249070 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:12.249170 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:12.249254 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:12.249667 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:12.250519 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:12.251760 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:12.252153 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:12.252242 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:12.253169 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:12.253177 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:12.253193 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:12.253789 (+ 596us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:12.253834 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:12.253859 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:12.253941 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:12.254038 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 15:12:12.254129 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 15:12:12.255023 23683 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:12:12.342480 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 23496
I20260809 15:12:12.365523 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 22987
I20260809 15:12:12.366541 23074 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:12:12.475458 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 22987
2026-08-09T15:12:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2701 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:12:12 dist-test-slave-nj21 krb5kdc[23755](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:12:12 dist-test-slave-nj21 krb5kdc[23755](info): set up 2 sockets
Aug 09 15:12:12 dist-test-slave-nj21 krb5kdc[23755](info): commencing operation
krb5kdc: starting...
W20260809 15:12:13.701282 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:12:13 dist-test-slave-nj21 krb5kdc[23755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:12:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:12:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
I20260809 15:12:13.986918 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:36081
--webserver_interface=127.24.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:36437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:36081
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:12:14.327562 23774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:14.327863 23774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:14.327998 23774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:14.335373 23774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:12:14.335484 23774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:14.335564 23774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:14.335645 23774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:12:14.335728 23774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:12:14.345175 23774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:36437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:36081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:36081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:14.346451 23774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:14.347740 23774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:14.359126 23781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:14.359076 23785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:14.359172 23783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:14.359989 23774 server_base.cc:1034] running on GCE node
I20260809 15:12:14.360688 23774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:14.361980 23774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:14.363186 23774 hybrid_clock.cc:648] HybridClock initialized: now 1786288334363142 us; error 27 us; skew 500 ppm
I20260809 15:12:14.363531 23774 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:12:14 dist-test-slave-nj21 krb5kdc[23755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288334, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:12:14.366478 23774 init.cc:377] Logged in from keytab as kudu/127.24.183.254@KRBTEST.COM (short username kudu)
I20260809 15:12:14.367980 23774 webserver.cc:460] Webserver started at http://127.24.183.254:35207/ using document root <none> and password file <none>
I20260809 15:12:14.368623 23774 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:14.368733 23774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:14.369032 23774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:14.372036 23774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "987a1f417c1e43539dc9210e53707c14"
format_stamp: "Formatted at 2026-08-09 15:12:14 on dist-test-slave-nj21"
server_key: "91abe2108a70a2c8bc268a022f7e88b2"
server_key_iv: "e280f9fbb2691a6bdb2c70a0d3274a9d"
server_key_version: "encryptionkey@0"
I20260809 15:12:14.372756 23774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "987a1f417c1e43539dc9210e53707c14"
format_stamp: "Formatted at 2026-08-09 15:12:14 on dist-test-slave-nj21"
server_key: "91abe2108a70a2c8bc268a022f7e88b2"
server_key_iv: "e280f9fbb2691a6bdb2c70a0d3274a9d"
server_key_version: "encryptionkey@0"
I20260809 15:12:14.377710 23774 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 15:12:14.380980 23797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:14.382236 23774 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:12:14.382414 23774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "987a1f417c1e43539dc9210e53707c14"
format_stamp: "Formatted at 2026-08-09 15:12:14 on dist-test-slave-nj21"
server_key: "91abe2108a70a2c8bc268a022f7e88b2"
server_key_iv: "e280f9fbb2691a6bdb2c70a0d3274a9d"
server_key_version: "encryptionkey@0"
I20260809 15:12:14.382591 23774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:14.402771 23774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:14.406417 23774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:14.406675 23774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:14.424144 23774 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:36081
I20260809 15:12:14.424173 23872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:36081 every 8 connection(s)
I20260809 15:12:14.425437 23774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:12:14.428004 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 23774
I20260809 15:12:14.428187 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:12:14.428617 25311 external_mini_cluster.cc:1428] Setting key bb81c83aa05a88e2960ca0280554a298
I20260809 15:12:14.429106 23874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:14.442947 23874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 987a1f417c1e43539dc9210e53707c14: Bootstrap starting.
Aug 09 15:12:14 dist-test-slave-nj21 krb5kdc[23755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:12:14.446599 23874 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 987a1f417c1e43539dc9210e53707c14: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:14.447954 23874 log.cc:824] T 00000000000000000000000000000000 P 987a1f417c1e43539dc9210e53707c14: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:14.451270 23874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 987a1f417c1e43539dc9210e53707c14: No bootstrap required, opened a new log
I20260809 15:12:14.452646 23878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:14.430430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40940 (local address 127.24.183.254:36081)
0809 15:12:14.431210 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:14.431224 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:14.431273 (+ 49us) server_negotiation.cc:407] Connection header received
0809 15:12:14.433772 (+ 2499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:14.433801 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:14.434263 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:14.434683 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:14.436017 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:14.437945 (+ 1928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:14.439255 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:14.440320 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:14.440530 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:14.444583 (+ 4053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:14.444593 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:14.444611 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:12:14.444677 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:14.448412 (+ 3735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:14.449188 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:14.449194 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:14.449199 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:14.449381 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:14.449733 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:14.449738 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:14.449741 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:14.450396 (+ 655us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:14.450678 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:14.450976 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 15:12:14.451370 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":322,"threads_started":1}
I20260809 15:12:14.459363 23874 raft_consensus.cc:348] T 00000000000000000000000000000000 P 987a1f417c1e43539dc9210e53707c14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "987a1f417c1e43539dc9210e53707c14" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36081 } }
I20260809 15:12:14.459709 23874 raft_consensus.cc:374] T 00000000000000000000000000000000 P 987a1f417c1e43539dc9210e53707c14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:14.459797 23874 raft_consensus.cc:729] T 00000000000000000000000000000000 P 987a1f417c1e43539dc9210e53707c14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 987a1f417c1e43539dc9210e53707c14, State: Initialized, Role: FOLLOWER
I20260809 15:12:14.460286 23874 consensus_queue.cc:260] T 00000000000000000000000000000000 P 987a1f417c1e43539dc9210e53707c14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "987a1f417c1e43539dc9210e53707c14" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36081 } }
I20260809 15:12:14.460498 23874 raft_consensus.cc:388] T 00000000000000000000000000000000 P 987a1f417c1e43539dc9210e53707c14 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:12:14.460592 23874 raft_consensus.cc:482] T 00000000000000000000000000000000 P 987a1f417c1e43539dc9210e53707c14 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:12:14.460716 23874 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 987a1f417c1e43539dc9210e53707c14 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:14.462929 23874 raft_consensus.cc:504] T 00000000000000000000000000000000 P 987a1f417c1e43539dc9210e53707c14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "987a1f417c1e43539dc9210e53707c14" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36081 } }
I20260809 15:12:14.463351 23874 leader_election.cc:302] T 00000000000000000000000000000000 P 987a1f417c1e43539dc9210e53707c14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 987a1f417c1e43539dc9210e53707c14; no voters:
I20260809 15:12:14.463973 23874 leader_election.cc:288] T 00000000000000000000000000000000 P 987a1f417c1e43539dc9210e53707c14 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:12:14.464082 23883 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 987a1f417c1e43539dc9210e53707c14 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:12:14.464713 23883 raft_consensus.cc:686] T 00000000000000000000000000000000 P 987a1f417c1e43539dc9210e53707c14 [term 1 LEADER]: Becoming Leader. State: Replica: 987a1f417c1e43539dc9210e53707c14, State: Running, Role: LEADER
I20260809 15:12:14.465278 23883 consensus_queue.cc:237] T 00000000000000000000000000000000 P 987a1f417c1e43539dc9210e53707c14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "987a1f417c1e43539dc9210e53707c14" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36081 } }
I20260809 15:12:14.465773 23874 sys_catalog.cc:564] T 00000000000000000000000000000000 P 987a1f417c1e43539dc9210e53707c14 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:12:14.467957 23884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 987a1f417c1e43539dc9210e53707c14 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "987a1f417c1e43539dc9210e53707c14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "987a1f417c1e43539dc9210e53707c14" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36081 } } }
I20260809 15:12:14.468334 23884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 987a1f417c1e43539dc9210e53707c14 [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:14.468791 23887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 987a1f417c1e43539dc9210e53707c14 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 987a1f417c1e43539dc9210e53707c14. Latest consensus state: current_term: 1 leader_uuid: "987a1f417c1e43539dc9210e53707c14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "987a1f417c1e43539dc9210e53707c14" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36081 } } }
I20260809 15:12:14.469118 23887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 987a1f417c1e43539dc9210e53707c14 [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:14.470244 23895 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:12:14.475184 23895 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:12:14.483438 23895 catalog_manager.cc:1347] Generated new cluster ID: 34617eea589f497c9829fddaa45d8281
I20260809 15:12:14.483546 23895 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:12:14.517627 23895 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:12:14.519493 23895 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:12:14.527077 23895 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 987a1f417c1e43539dc9210e53707c14: Generated new TSK 0
I20260809 15:12:14.528040 23895 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab.
Entry for principal kudu/127.24.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab.
I20260809 15:12:14.594607 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.193:0
--local_ip_for_outbound_sockets=127.24.183.193
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:36081
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:36437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:12:14.927222 23911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:14.927480 23911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:14.927615 23911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:14.934872 23911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:14.934974 23911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:14.935127 23911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.193
I20260809 15:12:14.944586 23911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:36437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.193.keytab
--principal=kudu/127.24.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:36081
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.193
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:14.945830 23911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:14.947098 23911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:14.959901 23923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:14.960083 23919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:14.960078 23917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:14.960433 23911 server_base.cc:1034] running on GCE node
I20260809 15:12:14.961210 23911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:14.962208 23911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:14.963430 23911 hybrid_clock.cc:648] HybridClock initialized: now 1786288334963384 us; error 30 us; skew 500 ppm
I20260809 15:12:14.963809 23911 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:12:14 dist-test-slave-nj21 krb5kdc[23755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288334, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:12:14.967051 23911 init.cc:377] Logged in from keytab as kudu/127.24.183.193@KRBTEST.COM (short username kudu)
I20260809 15:12:14.968693 23911 webserver.cc:460] Webserver started at http://127.24.183.193:40239/ using document root <none> and password file <none>
I20260809 15:12:14.969906 23911 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:14.970055 23911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:14.970404 23911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:14.973564 23911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-0/data/instance:
uuid: "bae1ba804f2049179677bee582ccb65e"
format_stamp: "Formatted at 2026-08-09 15:12:14 on dist-test-slave-nj21"
server_key: "20eaf2c90e3c253f640aec6ae287369a"
server_key_iv: "0039bd2c90b493346e0f21ccd80ef03d"
server_key_version: "encryptionkey@0"
I20260809 15:12:14.974356 23911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance:
uuid: "bae1ba804f2049179677bee582ccb65e"
format_stamp: "Formatted at 2026-08-09 15:12:14 on dist-test-slave-nj21"
server_key: "20eaf2c90e3c253f640aec6ae287369a"
server_key_iv: "0039bd2c90b493346e0f21ccd80ef03d"
server_key_version: "encryptionkey@0"
I20260809 15:12:14.979919 23911 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260809 15:12:14.983469 23932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:14.984731 23911 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:12:14.984884 23911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-0/wal
uuid: "bae1ba804f2049179677bee582ccb65e"
format_stamp: "Formatted at 2026-08-09 15:12:14 on dist-test-slave-nj21"
server_key: "20eaf2c90e3c253f640aec6ae287369a"
server_key_iv: "0039bd2c90b493346e0f21ccd80ef03d"
server_key_version: "encryptionkey@0"
I20260809 15:12:14.985069 23911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:15.020696 23911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:15.024544 23911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:15.024816 23911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:15.025861 23911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:12:15.027545 23911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:12:15.027627 23911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:15.027779 23911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:12:15.027833 23911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:15.058591 23911 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.193:43469
I20260809 15:12:15.058636 24101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.193:43469 every 8 connection(s)
I20260809 15:12:15.059907 23911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
I20260809 15:12:15.062181 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 23911
I20260809 15:12:15.062367 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance
I20260809 15:12:15.062942 25311 external_mini_cluster.cc:1428] Setting key 0ac0d8e324160f154e20c640c8ad1cb0
Aug 09 15:12:15 dist-test-slave-nj21 krb5kdc[23755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288334, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.193@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:12:15.080808 24104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:15.062450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:43679 (local address 127.24.183.254:36081)
0809 15:12:15.062964 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:15.062971 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:15.063932 (+ 961us) server_negotiation.cc:407] Connection header received
0809 15:12:15.066081 (+ 2149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:15.066087 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:15.066169 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:15.066263 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:15.067865 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:15.068851 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:15.070578 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:15.071101 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:15.071222 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:15.074751 (+ 3529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:15.074757 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:15.074762 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:12:15.074800 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:15.077756 (+ 2956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:15.078468 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:15.078473 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:15.078477 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:15.078628 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:15.079088 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:15.079091 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:15.079093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:15.079361 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:15.079471 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:15.080352 (+ 881us) server_negotiation.cc:299] Negotiation successful
0809 15:12:15.080529 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":245,"threads_started":1}
I20260809 15:12:15.082126 24105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:15.062945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:36081 (local address 127.24.183.193:43679)
0809 15:12:15.063762 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:15.063813 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:15.065827 (+ 2014us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:15.066542 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:15.066553 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:15.067093 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:12:15.067649 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:15.067666 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:15.069017 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:15.069023 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:15.070435 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:15.070448 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:15.071327 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:15.071332 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:15.071545 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:15.072184 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:12:15.072211 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:12:15.074531 (+ 2320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:12:15.077895 (+ 3364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:12:15.077902 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:12:15.077916 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:12:15.078342 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:12:15.078722 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:12:15.078727 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:12:15.078731 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:12:15.078991 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:12:15.079477 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:12:15.079485 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:12:15.079967 (+ 482us) client_negotiation.cc:769] Sending connection context
0809 15:12:15.080253 (+ 286us) client_negotiation.cc:240] Negotiation successful
0809 15:12:15.080775 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":314,"threads_started":1}
I20260809 15:12:15.083639 24102 heartbeater.cc:347] Connected to a master server at 127.24.183.254:36081
I20260809 15:12:15.084008 24102 heartbeater.cc:464] Registering TS with master...
I20260809 15:12:15.084844 24102 heartbeater.cc:511] Master 127.24.183.254:36081 requested a full tablet report, sending...
I20260809 15:12:15.087189 23824 ts_manager.cc:194] Registered new tserver with Master: bae1ba804f2049179677bee582ccb65e (127.24.183.193:43469)
I20260809 15:12:15.088781 23824 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.193@KRBTEST.COM'} at 127.24.183.193:43679
WARNING: no policy specified for kudu/127.24.183.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab.
Entry for principal kudu/127.24.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab.
I20260809 15:12:15.130115 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.194:0
--local_ip_for_outbound_sockets=127.24.183.194
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:36081
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:36437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:12:15.467914 24111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:15.468164 24111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:15.468312 24111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:15.475487 24111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:15.475589 24111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:15.475769 24111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.194
I20260809 15:12:15.485481 24111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:36437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.194.keytab
--principal=kudu/127.24.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:36081
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.194
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:15.486778 24111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:15.488155 24111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:15.500718 24118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:15.500679 24119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:15.500777 24121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:15.501776 24111 server_base.cc:1034] running on GCE node
I20260809 15:12:15.502357 24111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:15.503194 24111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:15.504387 24111 hybrid_clock.cc:648] HybridClock initialized: now 1786288335504336 us; error 42 us; skew 500 ppm
I20260809 15:12:15.504666 24111 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:12:15 dist-test-slave-nj21 krb5kdc[23755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288335, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:12:15.507598 24111 init.cc:377] Logged in from keytab as kudu/127.24.183.194@KRBTEST.COM (short username kudu)
I20260809 15:12:15.508946 24111 webserver.cc:460] Webserver started at http://127.24.183.194:41345/ using document root <none> and password file <none>
I20260809 15:12:15.509588 24111 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:15.509703 24111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:15.509994 24111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:15.512789 24111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-1/data/instance:
uuid: "7674f79eb3f2413ea7c46a9ac8d8e639"
format_stamp: "Formatted at 2026-08-09 15:12:15 on dist-test-slave-nj21"
server_key: "7cd21105e9ab3c4482588af15f9ffd32"
server_key_iv: "0136897745aacf6e51bd6ccfbc8e8566"
server_key_version: "encryptionkey@0"
I20260809 15:12:15.513515 24111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance:
uuid: "7674f79eb3f2413ea7c46a9ac8d8e639"
format_stamp: "Formatted at 2026-08-09 15:12:15 on dist-test-slave-nj21"
server_key: "7cd21105e9ab3c4482588af15f9ffd32"
server_key_iv: "0136897745aacf6e51bd6ccfbc8e8566"
server_key_version: "encryptionkey@0"
I20260809 15:12:15.518522 24111 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 15:12:15.521764 24130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:15.522975 24111 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260809 15:12:15.523120 24111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-1/wal
uuid: "7674f79eb3f2413ea7c46a9ac8d8e639"
format_stamp: "Formatted at 2026-08-09 15:12:15 on dist-test-slave-nj21"
server_key: "7cd21105e9ab3c4482588af15f9ffd32"
server_key_iv: "0136897745aacf6e51bd6ccfbc8e8566"
server_key_version: "encryptionkey@0"
I20260809 15:12:15.523289 24111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:15.549521 24111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:15.553136 24111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:15.553399 24111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:15.554294 24111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:12:15.555572 24111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:12:15.555642 24111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:15.555804 24111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:12:15.555855 24111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:15.586956 24111 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.194:33751
I20260809 15:12:15.587004 24292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.194:33751 every 8 connection(s)
I20260809 15:12:15.588344 24111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
I20260809 15:12:15.596983 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 24111
I20260809 15:12:15.597189 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance
I20260809 15:12:15.597741 25311 external_mini_cluster.cc:1428] Setting key 56f83b2fc381166ea872a0db75b5d718
Aug 09 15:12:15 dist-test-slave-nj21 krb5kdc[23755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288335, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.194@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:12:15.608732 24295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:15.590974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:54601 (local address 127.24.183.254:36081)
0809 15:12:15.591438 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:15.591445 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:15.592454 (+ 1009us) server_negotiation.cc:407] Connection header received
0809 15:12:15.594246 (+ 1792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:15.594250 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:15.594323 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:15.594423 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:15.595857 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:15.596601 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:15.598689 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:15.599265 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:15.599395 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:15.603278 (+ 3883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:15.603283 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:15.603286 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:12:15.603319 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:15.605773 (+ 2454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:15.606571 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:15.606575 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:15.606577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:15.606711 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:15.607121 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:15.607123 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:15.607125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:15.607397 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:15.607506 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:15.608283 (+ 777us) server_negotiation.cc:299] Negotiation successful
0809 15:12:15.608463 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":201,"threads_started":1}
I20260809 15:12:15.610049 24296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:15.591521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:36081 (local address 127.24.183.194:54601)
0809 15:12:15.592283 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:15.592322 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:15.594011 (+ 1689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:15.594610 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:15.594617 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:15.595102 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:12:15.595689 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:15.595716 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:15.596746 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:15.596751 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:15.598536 (+ 1785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:15.598560 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:15.599485 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:15.599490 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:15.599774 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:15.600426 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:12:15.600456 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:12:15.603055 (+ 2599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:12:15.605924 (+ 2869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:12:15.605932 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:12:15.605947 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:12:15.606473 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:12:15.606803 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:12:15.606808 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:12:15.606810 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:12:15.607033 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:12:15.607507 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:12:15.607514 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:12:15.607921 (+ 407us) client_negotiation.cc:769] Sending connection context
0809 15:12:15.608200 (+ 279us) client_negotiation.cc:240] Negotiation successful
0809 15:12:15.608717 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":314,"threads_started":1}
I20260809 15:12:15.611425 24293 heartbeater.cc:347] Connected to a master server at 127.24.183.254:36081
I20260809 15:12:15.611773 24293 heartbeater.cc:464] Registering TS with master...
I20260809 15:12:15.612591 24293 heartbeater.cc:511] Master 127.24.183.254:36081 requested a full tablet report, sending...
I20260809 15:12:15.614044 23824 ts_manager.cc:194] Registered new tserver with Master: 7674f79eb3f2413ea7c46a9ac8d8e639 (127.24.183.194:33751)
I20260809 15:12:15.614850 23824 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.194@KRBTEST.COM'} at 127.24.183.194:54601
WARNING: no policy specified for kudu/127.24.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab.
Entry for principal kudu/127.24.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab.
I20260809 15:12:15.660339 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.195:0
--local_ip_for_outbound_sockets=127.24.183.195
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:36081
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:36437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:12:16.009090 24299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:16.009375 24299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:16.009534 24299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:16.016826 24299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:16.016933 24299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:16.017097 24299 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.195
I20260809 15:12:16.026760 24299 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:36437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.195.keytab
--principal=kudu/127.24.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:36081
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.195
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:16.028106 24299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:16.029378 24299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:16.042147 24307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:16.042088 24311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:16.042156 24308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:16.042944 24299 server_base.cc:1034] running on GCE node
I20260809 15:12:16.043618 24299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:16.044597 24299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:16.045823 24299 hybrid_clock.cc:648] HybridClock initialized: now 1786288336045769 us; error 36 us; skew 500 ppm
I20260809 15:12:16.046139 24299 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:12:16 dist-test-slave-nj21 krb5kdc[23755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288336, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:12:16.049139 24299 init.cc:377] Logged in from keytab as kudu/127.24.183.195@KRBTEST.COM (short username kudu)
I20260809 15:12:16.050552 24299 webserver.cc:460] Webserver started at http://127.24.183.195:39635/ using document root <none> and password file <none>
I20260809 15:12:16.051182 24299 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:16.051296 24299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:16.051591 24299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:16.054630 24299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-2/data/instance:
uuid: "ee7e2e5f6c0a4e438f94772ccbd76d91"
format_stamp: "Formatted at 2026-08-09 15:12:16 on dist-test-slave-nj21"
server_key: "a16e9b37f749264746cf269cdb6e04bc"
server_key_iv: "6cbe04e45c91251b4aa68f1fc7e410d1"
server_key_version: "encryptionkey@0"
I20260809 15:12:16.055361 24299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance:
uuid: "ee7e2e5f6c0a4e438f94772ccbd76d91"
format_stamp: "Formatted at 2026-08-09 15:12:16 on dist-test-slave-nj21"
server_key: "a16e9b37f749264746cf269cdb6e04bc"
server_key_iv: "6cbe04e45c91251b4aa68f1fc7e410d1"
server_key_version: "encryptionkey@0"
I20260809 15:12:16.061043 24299 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 15:12:16.065128 24322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:16.066382 24299 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.003s
I20260809 15:12:16.066532 24299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-2/wal
uuid: "ee7e2e5f6c0a4e438f94772ccbd76d91"
format_stamp: "Formatted at 2026-08-09 15:12:16 on dist-test-slave-nj21"
server_key: "a16e9b37f749264746cf269cdb6e04bc"
server_key_iv: "6cbe04e45c91251b4aa68f1fc7e410d1"
server_key_version: "encryptionkey@0"
I20260809 15:12:16.066694 24299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:16.077565 24299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:16.081168 24299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:16.081413 24299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:16.082284 24299 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:12:16.083503 24299 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:12:16.083573 24299 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:16.083710 24299 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:12:16.083766 24299 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260809 15:12:16.091773 24102 heartbeater.cc:503] Master 127.24.183.254:36081 was elected leader, sending a full tablet report...
I20260809 15:12:16.115216 24299 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.195:40149
I20260809 15:12:16.115267 24484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.195:40149 every 8 connection(s)
I20260809 15:12:16.116557 24299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
I20260809 15:12:16.119009 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 24299
I20260809 15:12:16.119182 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance
I20260809 15:12:16.119807 25311 external_mini_cluster.cc:1428] Setting key 8b44b11ddd630c6d6ce50cb6f1442e96
Aug 09 15:12:16 dist-test-slave-nj21 krb5kdc[23755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288336, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.195@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:12:16.135979 24487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:16.119099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:35683 (local address 127.24.183.254:36081)
0809 15:12:16.119641 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:16.119647 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:16.120599 (+ 952us) server_negotiation.cc:407] Connection header received
0809 15:12:16.122757 (+ 2158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:16.122761 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:16.122846 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:16.122935 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:16.124370 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.125152 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.126648 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.127050 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.127141 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:16.130383 (+ 3242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:16.130388 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:16.130398 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:12:16.130431 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:16.132824 (+ 2393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:16.133600 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:16.133604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:16.133606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:16.133759 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:16.134371 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:16.134374 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:16.134376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:16.134697 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:16.134825 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:16.135484 (+ 659us) server_negotiation.cc:299] Negotiation successful
0809 15:12:16.135648 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":253,"threads_started":1}
I20260809 15:12:16.136852 24488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:16.119631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:36081 (local address 127.24.183.195:35683)
0809 15:12:16.120424 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:16.120464 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:16.122525 (+ 2061us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:16.123122 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:16.123129 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:16.123611 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:12:16.124220 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:16.124240 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.125267 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.125271 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:16.126494 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:16.126512 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.127224 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.127228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:16.127423 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:16.128039 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:12:16.128061 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:12:16.130202 (+ 2141us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:12:16.132957 (+ 2755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:12:16.132965 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:12:16.132979 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:12:16.133492 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:12:16.133864 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:12:16.133869 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:12:16.133872 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:12:16.134123 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:12:16.134834 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:12:16.134841 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:12:16.135152 (+ 311us) client_negotiation.cc:769] Sending connection context
0809 15:12:16.135379 (+ 227us) client_negotiation.cc:240] Negotiation successful
0809 15:12:16.135793 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":296,"threads_started":1}
I20260809 15:12:16.138091 24485 heartbeater.cc:347] Connected to a master server at 127.24.183.254:36081
I20260809 15:12:16.138383 24485 heartbeater.cc:464] Registering TS with master...
I20260809 15:12:16.139065 24485 heartbeater.cc:511] Master 127.24.183.254:36081 requested a full tablet report, sending...
I20260809 15:12:16.140415 23824 ts_manager.cc:194] Registered new tserver with Master: ee7e2e5f6c0a4e438f94772ccbd76d91 (127.24.183.195:40149)
I20260809 15:12:16.141206 23824 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.183.195@KRBTEST.COM'} at 127.24.183.195:35683
I20260809 15:12:16.146890 25311 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:12:16.163371 24487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:16.151390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36608 (local address 127.24.183.254:36081)
0809 15:12:16.151631 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:16.151638 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:16.152171 (+ 533us) server_negotiation.cc:407] Connection header received
0809 15:12:16.152304 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:16.152307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:16.152396 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:16.152470 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:16.153843 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.154525 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.155770 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.156171 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.156263 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:16.158572 (+ 2309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:16.158576 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:16.158580 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:12:16.158610 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:16.160908 (+ 2298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:16.161506 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:16.161510 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:16.161512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:16.161646 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:16.161989 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:16.161993 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:16.161995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:16.162309 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:16.162549 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:16.162862 (+ 313us) server_negotiation.cc:299] Negotiation successful
0809 15:12:16.163095 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 15:12:16.167608 23824 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36608:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:12:16.174887 23824 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:12:16.209300 24512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:16.188435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41634 (local address 127.24.183.194:33751)
0809 15:12:16.196956 (+ 8521us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:16.196966 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:16.202072 (+ 5106us) server_negotiation.cc:407] Connection header received
0809 15:12:16.202184 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:16.202190 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:16.202413 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:16.202612 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:12:16.203552 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.205075 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.206939 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.208242 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.208392 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:16.208567 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:16.208813 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 15:12:16.208984 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8311,"thread_start_us":8153,"threads_started":1}
I20260809 15:12:16.209300 24502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:16.187043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:33751 (local address 127.0.0.1:41634)
0809 15:12:16.201841 (+ 14798us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:16.201857 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:16.201968 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:16.202914 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:16.202919 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:16.202944 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:12:16.203359 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:16.203372 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.205203 (+ 1831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.205207 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:16.206787 (+ 1580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:16.206799 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.208430 (+ 1631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.208437 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:16.208613 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:16.208652 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 15:12:16.208759 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 15:12:16.208936 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14702,"thread_start_us":13556,"threads_started":1}
I20260809 15:12:16.210570 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:16.187881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36086 (local address 127.24.183.195:40149)
0809 15:12:16.202151 (+ 14270us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:16.202158 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:16.202186 (+ 28us) server_negotiation.cc:407] Connection header received
0809 15:12:16.202272 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:16.202278 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:16.202438 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:16.202612 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:12:16.203312 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.204910 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.207609 (+ 2699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.208987 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.209175 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:16.209343 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:16.210102 (+ 759us) server_negotiation.cc:299] Negotiation successful
0809 15:12:16.210256 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14115,"thread_start_us":13921,"threads_started":1}
I20260809 15:12:16.211211 24507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:16.187094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36624 (local address 127.24.183.193:43469)
0809 15:12:16.201590 (+ 14496us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:16.201598 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:16.201628 (+ 30us) server_negotiation.cc:407] Connection header received
0809 15:12:16.201724 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:16.201731 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:16.202021 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:16.202198 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:12:16.202911 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.204483 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.207259 (+ 2776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.209002 (+ 1743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.209176 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:16.209330 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:16.210836 (+ 1506us) server_negotiation.cc:299] Negotiation successful
0809 15:12:16.210967 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14324,"thread_start_us":14165,"threads_started":1}
I20260809 15:12:16.210616 24503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:16.186840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:43469 (local address 127.0.0.1:36624)
0809 15:12:16.198608 (+ 11768us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:16.198657 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:16.198853 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:16.202307 (+ 3454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:16.202314 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:16.202351 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:12:16.202693 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:16.202705 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.204640 (+ 1935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.204643 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:16.207085 (+ 2442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:16.207100 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.209566 (+ 2466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.209573 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:16.209868 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:16.209938 (+ 70us) client_negotiation.cc:769] Sending connection context
0809 15:12:16.210028 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 15:12:16.210163 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11488,"mutex_wait_us":149,"thread_start_us":15216,"threads_started":1}
I20260809 15:12:16.210639 24504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:16.186980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:40149 (local address 127.0.0.1:36086)
0809 15:12:16.200867 (+ 13887us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:16.200900 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:16.201012 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:16.202691 (+ 1679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:16.202696 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:16.202720 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:12:16.203132 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:16.203141 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.205043 (+ 1902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.205049 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:16.207427 (+ 2378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:16.207441 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.209554 (+ 2113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.209561 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:16.209893 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:16.209945 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 15:12:16.210028 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 15:12:16.210163 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13763,"mutex_wait_us":205,"thread_start_us":11199,"threads_started":1}
I20260809 15:12:16.213451 24197 tablet_service.cc:1467] Processing CreateTablet for tablet a3212cbac7024f83976255912a5d23af (DEFAULT_TABLE table=test-table [id=523d8f00b70140afbe6ae3a960bc4a28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:12:16.215057 24197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3212cbac7024f83976255912a5d23af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:16.215487 24001 tablet_service.cc:1467] Processing CreateTablet for tablet a3212cbac7024f83976255912a5d23af (DEFAULT_TABLE table=test-table [id=523d8f00b70140afbe6ae3a960bc4a28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:12:16.217283 24001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3212cbac7024f83976255912a5d23af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:16.217325 24390 tablet_service.cc:1467] Processing CreateTablet for tablet a3212cbac7024f83976255912a5d23af (DEFAULT_TABLE table=test-table [id=523d8f00b70140afbe6ae3a960bc4a28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:12:16.218379 24390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3212cbac7024f83976255912a5d23af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:16.228590 24514 tablet_bootstrap.cc:492] T a3212cbac7024f83976255912a5d23af P ee7e2e5f6c0a4e438f94772ccbd76d91: Bootstrap starting.
I20260809 15:12:16.230783 24515 tablet_bootstrap.cc:492] T a3212cbac7024f83976255912a5d23af P 7674f79eb3f2413ea7c46a9ac8d8e639: Bootstrap starting.
I20260809 15:12:16.232138 24516 tablet_bootstrap.cc:492] T a3212cbac7024f83976255912a5d23af P bae1ba804f2049179677bee582ccb65e: Bootstrap starting.
I20260809 15:12:16.232843 24514 tablet_bootstrap.cc:654] T a3212cbac7024f83976255912a5d23af P ee7e2e5f6c0a4e438f94772ccbd76d91: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:16.234124 24514 log.cc:824] T a3212cbac7024f83976255912a5d23af P ee7e2e5f6c0a4e438f94772ccbd76d91: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:16.234143 24515 tablet_bootstrap.cc:654] T a3212cbac7024f83976255912a5d23af P 7674f79eb3f2413ea7c46a9ac8d8e639: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:16.235251 24516 tablet_bootstrap.cc:654] T a3212cbac7024f83976255912a5d23af P bae1ba804f2049179677bee582ccb65e: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:16.235397 24515 log.cc:824] T a3212cbac7024f83976255912a5d23af P 7674f79eb3f2413ea7c46a9ac8d8e639: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:16.236413 24516 log.cc:824] T a3212cbac7024f83976255912a5d23af P bae1ba804f2049179677bee582ccb65e: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:16.238696 24515 tablet_bootstrap.cc:492] T a3212cbac7024f83976255912a5d23af P 7674f79eb3f2413ea7c46a9ac8d8e639: No bootstrap required, opened a new log
I20260809 15:12:16.238991 24514 tablet_bootstrap.cc:492] T a3212cbac7024f83976255912a5d23af P ee7e2e5f6c0a4e438f94772ccbd76d91: No bootstrap required, opened a new log
I20260809 15:12:16.239008 24515 ts_tablet_manager.cc:1397] T a3212cbac7024f83976255912a5d23af P 7674f79eb3f2413ea7c46a9ac8d8e639: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20260809 15:12:16.239210 24514 ts_tablet_manager.cc:1397] T a3212cbac7024f83976255912a5d23af P ee7e2e5f6c0a4e438f94772ccbd76d91: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.007s
I20260809 15:12:16.239303 24516 tablet_bootstrap.cc:492] T a3212cbac7024f83976255912a5d23af P bae1ba804f2049179677bee582ccb65e: No bootstrap required, opened a new log
I20260809 15:12:16.239563 24516 ts_tablet_manager.cc:1397] T a3212cbac7024f83976255912a5d23af P bae1ba804f2049179677bee582ccb65e: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260809 15:12:16.246001 24514 raft_consensus.cc:348] T a3212cbac7024f83976255912a5d23af P ee7e2e5f6c0a4e438f94772ccbd76d91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7674f79eb3f2413ea7c46a9ac8d8e639" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 33751 } } peers { permanent_uuid: "bae1ba804f2049179677bee582ccb65e" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 43469 } } peers { permanent_uuid: "ee7e2e5f6c0a4e438f94772ccbd76d91" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 40149 } }
I20260809 15:12:16.246347 24514 raft_consensus.cc:374] T a3212cbac7024f83976255912a5d23af P ee7e2e5f6c0a4e438f94772ccbd76d91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:16.246465 24514 raft_consensus.cc:729] T a3212cbac7024f83976255912a5d23af P ee7e2e5f6c0a4e438f94772ccbd76d91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee7e2e5f6c0a4e438f94772ccbd76d91, State: Initialized, Role: FOLLOWER
I20260809 15:12:16.247038 24514 consensus_queue.cc:260] T a3212cbac7024f83976255912a5d23af P ee7e2e5f6c0a4e438f94772ccbd76d91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7674f79eb3f2413ea7c46a9ac8d8e639" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 33751 } } peers { permanent_uuid: "bae1ba804f2049179677bee582ccb65e" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 43469 } } peers { permanent_uuid: "ee7e2e5f6c0a4e438f94772ccbd76d91" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 40149 } }
I20260809 15:12:16.248068 24485 heartbeater.cc:503] Master 127.24.183.254:36081 was elected leader, sending a full tablet report...
I20260809 15:12:16.249382 24515 raft_consensus.cc:348] T a3212cbac7024f83976255912a5d23af P 7674f79eb3f2413ea7c46a9ac8d8e639 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7674f79eb3f2413ea7c46a9ac8d8e639" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 33751 } } peers { permanent_uuid: "bae1ba804f2049179677bee582ccb65e" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 43469 } } peers { permanent_uuid: "ee7e2e5f6c0a4e438f94772ccbd76d91" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 40149 } }
I20260809 15:12:16.249469 24516 raft_consensus.cc:348] T a3212cbac7024f83976255912a5d23af P bae1ba804f2049179677bee582ccb65e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7674f79eb3f2413ea7c46a9ac8d8e639" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 33751 } } peers { permanent_uuid: "bae1ba804f2049179677bee582ccb65e" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 43469 } } peers { permanent_uuid: "ee7e2e5f6c0a4e438f94772ccbd76d91" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 40149 } }
I20260809 15:12:16.249825 24516 raft_consensus.cc:374] T a3212cbac7024f83976255912a5d23af P bae1ba804f2049179677bee582ccb65e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:16.249886 24515 raft_consensus.cc:374] T a3212cbac7024f83976255912a5d23af P 7674f79eb3f2413ea7c46a9ac8d8e639 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:16.249964 24516 raft_consensus.cc:729] T a3212cbac7024f83976255912a5d23af P bae1ba804f2049179677bee582ccb65e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bae1ba804f2049179677bee582ccb65e, State: Initialized, Role: FOLLOWER
I20260809 15:12:16.250027 24515 raft_consensus.cc:729] T a3212cbac7024f83976255912a5d23af P 7674f79eb3f2413ea7c46a9ac8d8e639 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7674f79eb3f2413ea7c46a9ac8d8e639, State: Initialized, Role: FOLLOWER
I20260809 15:12:16.250321 24514 ts_tablet_manager.cc:1428] T a3212cbac7024f83976255912a5d23af P ee7e2e5f6c0a4e438f94772ccbd76d91: Time spent starting tablet: real 0.011s user 0.007s sys 0.002s
I20260809 15:12:16.250626 24515 consensus_queue.cc:260] T a3212cbac7024f83976255912a5d23af P 7674f79eb3f2413ea7c46a9ac8d8e639 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7674f79eb3f2413ea7c46a9ac8d8e639" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 33751 } } peers { permanent_uuid: "bae1ba804f2049179677bee582ccb65e" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 43469 } } peers { permanent_uuid: "ee7e2e5f6c0a4e438f94772ccbd76d91" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 40149 } }
I20260809 15:12:16.250734 24516 consensus_queue.cc:260] T a3212cbac7024f83976255912a5d23af P bae1ba804f2049179677bee582ccb65e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7674f79eb3f2413ea7c46a9ac8d8e639" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 33751 } } peers { permanent_uuid: "bae1ba804f2049179677bee582ccb65e" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 43469 } } peers { permanent_uuid: "ee7e2e5f6c0a4e438f94772ccbd76d91" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 40149 } }
I20260809 15:12:16.251811 24293 heartbeater.cc:503] Master 127.24.183.254:36081 was elected leader, sending a full tablet report...
I20260809 15:12:16.253484 24516 ts_tablet_manager.cc:1428] T a3212cbac7024f83976255912a5d23af P bae1ba804f2049179677bee582ccb65e: Time spent starting tablet: real 0.014s user 0.009s sys 0.006s
I20260809 15:12:16.253705 24515 ts_tablet_manager.cc:1428] T a3212cbac7024f83976255912a5d23af P 7674f79eb3f2413ea7c46a9ac8d8e639: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
W20260809 15:12:16.312140 24103 tablet.cc:2369] T a3212cbac7024f83976255912a5d23af P bae1ba804f2049179677bee582ccb65e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:12:16.340411 24294 tablet.cc:2369] T a3212cbac7024f83976255912a5d23af P 7674f79eb3f2413ea7c46a9ac8d8e639: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:12:16.348197 24531 raft_consensus.cc:482] T a3212cbac7024f83976255912a5d23af P bae1ba804f2049179677bee582ccb65e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:12:16.348503 24531 raft_consensus.cc:504] T a3212cbac7024f83976255912a5d23af P bae1ba804f2049179677bee582ccb65e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7674f79eb3f2413ea7c46a9ac8d8e639" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 33751 } } peers { permanent_uuid: "bae1ba804f2049179677bee582ccb65e" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 43469 } } peers { permanent_uuid: "ee7e2e5f6c0a4e438f94772ccbd76d91" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 40149 } }
I20260809 15:12:16.350309 24531 leader_election.cc:288] T a3212cbac7024f83976255912a5d23af P bae1ba804f2049179677bee582ccb65e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7674f79eb3f2413ea7c46a9ac8d8e639 (127.24.183.194:33751), ee7e2e5f6c0a4e438f94772ccbd76d91 (127.24.183.195:40149)
I20260809 15:12:16.357016 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:16.351056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:53673 (local address 127.24.183.195:40149)
0809 15:12:16.351251 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:16.351256 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:16.351709 (+ 453us) server_negotiation.cc:407] Connection header received
0809 15:12:16.351939 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:16.351944 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:16.352021 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:16.352094 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:12:16.352773 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.353528 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.355303 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.356086 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.356205 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:16.356279 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:16.356632 (+ 353us) server_negotiation.cc:299] Negotiation successful
0809 15:12:16.356765 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 15:12:16.357064 24512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:16.350768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:52975 (local address 127.24.183.194:33751)
0809 15:12:16.350999 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:16.351007 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:16.351912 (+ 905us) server_negotiation.cc:407] Connection header received
0809 15:12:16.352065 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:16.352070 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:16.352151 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:16.352267 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:12:16.352856 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.353556 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.355322 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.356129 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.356280 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:16.356363 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:16.356727 (+ 364us) server_negotiation.cc:299] Negotiation successful
0809 15:12:16.356866 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 15:12:16.357111 24533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:16.350583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:40149 (local address 127.24.183.193:53673)
0809 15:12:16.351562 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:16.351582 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:16.351780 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:16.352125 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:16.352130 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:16.352162 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:12:16.352632 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:16.352643 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.353655 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.353661 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:16.355156 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:16.355168 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.356298 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.356305 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:16.356473 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:16.356497 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 15:12:16.356586 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 15:12:16.356715 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":825,"mutex_wait_us":20,"spinlock_wait_cycles":24832,"thread_start_us":676,"threads_started":1}
I20260809 15:12:16.357650 24534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:16.350684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:33751 (local address 127.24.183.193:52975)
0809 15:12:16.351807 (+ 1123us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:16.351820 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:16.351951 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:16.352285 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:16.352288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:16.352324 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:12:16.352704 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:16.352714 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.353691 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.353695 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:16.355168 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:16.355180 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.356353 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.356358 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:16.356519 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:16.356556 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 15:12:16.357256 (+ 700us) client_negotiation.cc:240] Negotiation successful
0809 15:12:16.357343 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":997,"mutex_wait_us":3,"thread_start_us":810,"threads_started":1}
I20260809 15:12:16.358451 24419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3212cbac7024f83976255912a5d23af" candidate_uuid: "bae1ba804f2049179677bee582ccb65e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee7e2e5f6c0a4e438f94772ccbd76d91" is_pre_election: true
I20260809 15:12:16.358675 24231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3212cbac7024f83976255912a5d23af" candidate_uuid: "bae1ba804f2049179677bee582ccb65e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7674f79eb3f2413ea7c46a9ac8d8e639" is_pre_election: true
I20260809 15:12:16.358934 24419 raft_consensus.cc:2454] T a3212cbac7024f83976255912a5d23af P ee7e2e5f6c0a4e438f94772ccbd76d91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bae1ba804f2049179677bee582ccb65e in term 0.
I20260809 15:12:16.359196 24231 raft_consensus.cc:2454] T a3212cbac7024f83976255912a5d23af P 7674f79eb3f2413ea7c46a9ac8d8e639 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bae1ba804f2049179677bee582ccb65e in term 0.
I20260809 15:12:16.359437 24526 raft_consensus.cc:482] T a3212cbac7024f83976255912a5d23af P 7674f79eb3f2413ea7c46a9ac8d8e639 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:12:16.359617 23933 leader_election.cc:302] T a3212cbac7024f83976255912a5d23af P bae1ba804f2049179677bee582ccb65e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bae1ba804f2049179677bee582ccb65e, ee7e2e5f6c0a4e438f94772ccbd76d91; no voters:
I20260809 15:12:16.359694 24526 raft_consensus.cc:504] T a3212cbac7024f83976255912a5d23af P 7674f79eb3f2413ea7c46a9ac8d8e639 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7674f79eb3f2413ea7c46a9ac8d8e639" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 33751 } } peers { permanent_uuid: "bae1ba804f2049179677bee582ccb65e" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 43469 } } peers { permanent_uuid: "ee7e2e5f6c0a4e438f94772ccbd76d91" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 40149 } }
I20260809 15:12:16.360046 24531 raft_consensus.cc:2781] T a3212cbac7024f83976255912a5d23af P bae1ba804f2049179677bee582ccb65e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:12:16.360157 24531 raft_consensus.cc:482] T a3212cbac7024f83976255912a5d23af P bae1ba804f2049179677bee582ccb65e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:12:16.360226 24531 raft_consensus.cc:3037] T a3212cbac7024f83976255912a5d23af P bae1ba804f2049179677bee582ccb65e [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:16.361184 24526 leader_election.cc:288] T a3212cbac7024f83976255912a5d23af P 7674f79eb3f2413ea7c46a9ac8d8e639 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bae1ba804f2049179677bee582ccb65e (127.24.183.193:43469), ee7e2e5f6c0a4e438f94772ccbd76d91 (127.24.183.195:40149)
I20260809 15:12:16.362991 24531 raft_consensus.cc:504] T a3212cbac7024f83976255912a5d23af P bae1ba804f2049179677bee582ccb65e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7674f79eb3f2413ea7c46a9ac8d8e639" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 33751 } } peers { permanent_uuid: "bae1ba804f2049179677bee582ccb65e" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 43469 } } peers { permanent_uuid: "ee7e2e5f6c0a4e438f94772ccbd76d91" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 40149 } }
I20260809 15:12:16.363791 24531 leader_election.cc:288] T a3212cbac7024f83976255912a5d23af P bae1ba804f2049179677bee582ccb65e [CANDIDATE]: Term 1 election: Requested vote from peers 7674f79eb3f2413ea7c46a9ac8d8e639 (127.24.183.194:33751), ee7e2e5f6c0a4e438f94772ccbd76d91 (127.24.183.195:40149)
I20260809 15:12:16.364142 24231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3212cbac7024f83976255912a5d23af" candidate_uuid: "bae1ba804f2049179677bee582ccb65e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7674f79eb3f2413ea7c46a9ac8d8e639"
I20260809 15:12:16.364347 24231 raft_consensus.cc:3037] T a3212cbac7024f83976255912a5d23af P 7674f79eb3f2413ea7c46a9ac8d8e639 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:16.365211 24419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3212cbac7024f83976255912a5d23af" candidate_uuid: "bae1ba804f2049179677bee582ccb65e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee7e2e5f6c0a4e438f94772ccbd76d91"
I20260809 15:12:16.365425 24419 raft_consensus.cc:3037] T a3212cbac7024f83976255912a5d23af P ee7e2e5f6c0a4e438f94772ccbd76d91 [term 0 FOLLOWER]: Advancing to term 1
W20260809 15:12:16.368108 24486 tablet.cc:2369] T a3212cbac7024f83976255912a5d23af P ee7e2e5f6c0a4e438f94772ccbd76d91: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:12:16.368801 24507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:16.362023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:50361 (local address 127.24.183.193:43469)
0809 15:12:16.362194 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:16.362199 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:16.362219 (+ 20us) server_negotiation.cc:407] Connection header received
0809 15:12:16.362269 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:16.362274 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:16.362347 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:16.362428 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:12:16.362867 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.363651 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.366584 (+ 2933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.367747 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.367859 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:16.367925 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:16.368464 (+ 539us) server_negotiation.cc:299] Negotiation successful
0809 15:12:16.368588 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 15:12:16.369095 24419 raft_consensus.cc:2454] T a3212cbac7024f83976255912a5d23af P ee7e2e5f6c0a4e438f94772ccbd76d91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bae1ba804f2049179677bee582ccb65e in term 1.
I20260809 15:12:16.369092 24231 raft_consensus.cc:2454] T a3212cbac7024f83976255912a5d23af P 7674f79eb3f2413ea7c46a9ac8d8e639 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bae1ba804f2049179677bee582ccb65e in term 1.
I20260809 15:12:16.369505 23936 leader_election.cc:302] T a3212cbac7024f83976255912a5d23af P bae1ba804f2049179677bee582ccb65e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7674f79eb3f2413ea7c46a9ac8d8e639, bae1ba804f2049179677bee582ccb65e; no voters:
I20260809 15:12:16.369571 24537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:16.361385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:43469 (local address 127.24.183.194:50361)
0809 15:12:16.362027 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:16.362045 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:16.362172 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:16.362456 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:16.362461 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:16.362479 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:12:16.362738 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:16.362745 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.364131 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.364135 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:16.366387 (+ 2252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:16.366413 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.368105 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.368111 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:16.368275 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:16.368305 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 15:12:16.368948 (+ 643us) client_negotiation.cc:240] Negotiation successful
0809 15:12:16.369087 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"mutex_wait_us":1,"thread_start_us":299,"threads_started":1}
I20260809 15:12:16.369896 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:16.361871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:42173 (local address 127.24.183.195:40149)
0809 15:12:16.362634 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:16.362639 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:16.362655 (+ 16us) server_negotiation.cc:407] Connection header received
0809 15:12:16.362706 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:16.362710 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:16.362771 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:16.362853 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:12:16.363761 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.364794 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.367453 (+ 2659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.368682 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.368822 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:16.368890 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:16.369538 (+ 648us) server_negotiation.cc:299] Negotiation successful
0809 15:12:16.369662 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651}
I20260809 15:12:16.370165 24531 raft_consensus.cc:2781] T a3212cbac7024f83976255912a5d23af P bae1ba804f2049179677bee582ccb65e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:12:16.370357 24034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3212cbac7024f83976255912a5d23af" candidate_uuid: "7674f79eb3f2413ea7c46a9ac8d8e639" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bae1ba804f2049179677bee582ccb65e" is_pre_election: true
I20260809 15:12:16.370705 24539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:16.361437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:40149 (local address 127.24.183.194:42173)
0809 15:12:16.362267 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:16.362282 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:16.362361 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:16.363127 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:16.363132 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:16.363158 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:12:16.363541 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:16.363554 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.364983 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.364988 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:16.367276 (+ 2288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:16.367291 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.369147 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.369154 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:16.369314 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:16.369357 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 15:12:16.370165 (+ 808us) client_negotiation.cc:240] Negotiation successful
0809 15:12:16.370296 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"mutex_wait_us":4,"thread_start_us":1489,"threads_started":1}
I20260809 15:12:16.370682 24531 raft_consensus.cc:686] T a3212cbac7024f83976255912a5d23af P bae1ba804f2049179677bee582ccb65e [term 1 LEADER]: Becoming Leader. State: Replica: bae1ba804f2049179677bee582ccb65e, State: Running, Role: LEADER
I20260809 15:12:16.371207 24531 consensus_queue.cc:237] T a3212cbac7024f83976255912a5d23af P bae1ba804f2049179677bee582ccb65e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7674f79eb3f2413ea7c46a9ac8d8e639" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 33751 } } peers { permanent_uuid: "bae1ba804f2049179677bee582ccb65e" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 43469 } } peers { permanent_uuid: "ee7e2e5f6c0a4e438f94772ccbd76d91" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 40149 } }
I20260809 15:12:16.371522 24419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3212cbac7024f83976255912a5d23af" candidate_uuid: "7674f79eb3f2413ea7c46a9ac8d8e639" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee7e2e5f6c0a4e438f94772ccbd76d91" is_pre_election: true
I20260809 15:12:16.371840 24419 raft_consensus.cc:2379] T a3212cbac7024f83976255912a5d23af P ee7e2e5f6c0a4e438f94772ccbd76d91 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7674f79eb3f2413ea7c46a9ac8d8e639 in current term 1: Already voted for candidate bae1ba804f2049179677bee582ccb65e in this term.
I20260809 15:12:16.375109 24135 leader_election.cc:302] T a3212cbac7024f83976255912a5d23af P 7674f79eb3f2413ea7c46a9ac8d8e639 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7674f79eb3f2413ea7c46a9ac8d8e639; no voters: bae1ba804f2049179677bee582ccb65e, ee7e2e5f6c0a4e438f94772ccbd76d91
I20260809 15:12:16.375435 24526 raft_consensus.cc:2726] T a3212cbac7024f83976255912a5d23af P 7674f79eb3f2413ea7c46a9ac8d8e639 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:12:16.375849 23822 catalog_manager.cc:5495] T a3212cbac7024f83976255912a5d23af P bae1ba804f2049179677bee582ccb65e reported cstate change: term changed from 0 to 1, leader changed from <none> to bae1ba804f2049179677bee582ccb65e (127.24.183.193). New cstate: current_term: 1 leader_uuid: "bae1ba804f2049179677bee582ccb65e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7674f79eb3f2413ea7c46a9ac8d8e639" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 33751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bae1ba804f2049179677bee582ccb65e" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 43469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee7e2e5f6c0a4e438f94772ccbd76d91" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 40149 } health_report { overall_health: UNKNOWN } } }
I20260809 15:12:16.398861 24507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:16.393937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36640 (local address 127.24.183.193:43469)
0809 15:12:16.394095 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:16.394100 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:16.394236 (+ 136us) server_negotiation.cc:407] Connection header received
0809 15:12:16.394415 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:16.394419 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:16.394499 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:16.394621 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:12:16.394909 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.395654 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.397002 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.397412 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.397497 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:16.397902 (+ 405us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:12:16.398407 (+ 505us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:12:16.398512 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:16.398567 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 15:12:16.398631 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 15:12:16.406456 24419 raft_consensus.cc:1261] T a3212cbac7024f83976255912a5d23af P ee7e2e5f6c0a4e438f94772ccbd76d91 [term 1 FOLLOWER]: Refusing update from remote peer bae1ba804f2049179677bee582ccb65e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:12:16.406517 24231 raft_consensus.cc:1261] T a3212cbac7024f83976255912a5d23af P 7674f79eb3f2413ea7c46a9ac8d8e639 [term 1 FOLLOWER]: Refusing update from remote peer bae1ba804f2049179677bee582ccb65e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:12:16.407290 24540 consensus_queue.cc:1035] T a3212cbac7024f83976255912a5d23af P bae1ba804f2049179677bee582ccb65e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7674f79eb3f2413ea7c46a9ac8d8e639" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 33751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:12:16.407619 24531 consensus_queue.cc:1035] T a3212cbac7024f83976255912a5d23af P bae1ba804f2049179677bee582ccb65e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7e2e5f6c0a4e438f94772ccbd76d91" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 40149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:12:16.432590 23821 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36608:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:12:16.432922 23821 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36608:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:12:16.437165 23821 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 987a1f417c1e43539dc9210e53707c14: Sending DeleteTablet for 3 replicas of tablet a3212cbac7024f83976255912a5d23af
I20260809 15:12:16.438227 24001 tablet_service.cc:1514] Processing DeleteTablet for tablet a3212cbac7024f83976255912a5d23af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:12:16 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:36624
I20260809 15:12:16.438243 24197 tablet_service.cc:1514] Processing DeleteTablet for tablet a3212cbac7024f83976255912a5d23af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:12:16 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:41634
I20260809 15:12:16.441275 24390 tablet_service.cc:1514] Processing DeleteTablet for tablet a3212cbac7024f83976255912a5d23af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:12:16 UTC) from {username='kudu', principal='kudu/127.24.183.254@KRBTEST.COM'} at 127.0.0.1:36086
I20260809 15:12:16.444383 24561 tablet_replica.cc:330] T a3212cbac7024f83976255912a5d23af P ee7e2e5f6c0a4e438f94772ccbd76d91: stopping tablet replica
I20260809 15:12:16.444813 24561 raft_consensus.cc:2229] T a3212cbac7024f83976255912a5d23af P ee7e2e5f6c0a4e438f94772ccbd76d91 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:12:16.445360 24561 raft_consensus.cc:2258] T a3212cbac7024f83976255912a5d23af P ee7e2e5f6c0a4e438f94772ccbd76d91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:12:16.445487 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 23911
I20260809 15:12:16.446280 24557 tablet_replica.cc:330] T a3212cbac7024f83976255912a5d23af P bae1ba804f2049179677bee582ccb65e: stopping tablet replica
I20260809 15:12:16.446663 24557 raft_consensus.cc:2229] T a3212cbac7024f83976255912a5d23af P bae1ba804f2049179677bee582ccb65e [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:12:16.447149 24557 raft_consensus.cc:2258] T a3212cbac7024f83976255912a5d23af P bae1ba804f2049179677bee582ccb65e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:12:16.448657 24561 ts_tablet_manager.cc:1905] T a3212cbac7024f83976255912a5d23af P ee7e2e5f6c0a4e438f94772ccbd76d91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:12:16.449259 24557 ts_tablet_manager.cc:1905] T a3212cbac7024f83976255912a5d23af P bae1ba804f2049179677bee582ccb65e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:12:16.455415 24561 ts_tablet_manager.cc:1918] T a3212cbac7024f83976255912a5d23af P ee7e2e5f6c0a4e438f94772ccbd76d91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:12:16.455565 24561 log.cc:1196] T a3212cbac7024f83976255912a5d23af P ee7e2e5f6c0a4e438f94772ccbd76d91: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-2/wal/wals/a3212cbac7024f83976255912a5d23af
I20260809 15:12:16.456108 24561 ts_tablet_manager.cc:1939] T a3212cbac7024f83976255912a5d23af P ee7e2e5f6c0a4e438f94772ccbd76d91: Deleting consensus metadata
I20260809 15:12:16.456372 24557 ts_tablet_manager.cc:1918] T a3212cbac7024f83976255912a5d23af P bae1ba804f2049179677bee582ccb65e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:12:16.456530 24557 log.cc:1196] T a3212cbac7024f83976255912a5d23af P bae1ba804f2049179677bee582ccb65e: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-0/wal/wals/a3212cbac7024f83976255912a5d23af
I20260809 15:12:16.457050 24557 ts_tablet_manager.cc:1939] T a3212cbac7024f83976255912a5d23af P bae1ba804f2049179677bee582ccb65e: Deleting consensus metadata
I20260809 15:12:16.457948 23798 catalog_manager.cc:4841] TS ee7e2e5f6c0a4e438f94772ccbd76d91 (127.24.183.195:40149): tablet a3212cbac7024f83976255912a5d23af (table test-table [id=523d8f00b70140afbe6ae3a960bc4a28]) successfully deleted
I20260809 15:12:16.458811 23806 catalog_manager.cc:4841] TS bae1ba804f2049179677bee582ccb65e (127.24.183.193:43469): tablet a3212cbac7024f83976255912a5d23af (table test-table [id=523d8f00b70140afbe6ae3a960bc4a28]) successfully deleted
I20260809 15:12:16.459336 24560 tablet_replica.cc:330] T a3212cbac7024f83976255912a5d23af P 7674f79eb3f2413ea7c46a9ac8d8e639: stopping tablet replica
I20260809 15:12:16.459760 24560 raft_consensus.cc:2229] T a3212cbac7024f83976255912a5d23af P 7674f79eb3f2413ea7c46a9ac8d8e639 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:12:16.460192 24560 raft_consensus.cc:2258] T a3212cbac7024f83976255912a5d23af P 7674f79eb3f2413ea7c46a9ac8d8e639 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:12:16.461918 24560 ts_tablet_manager.cc:1905] T a3212cbac7024f83976255912a5d23af P 7674f79eb3f2413ea7c46a9ac8d8e639: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:12:16.467124 24560 ts_tablet_manager.cc:1918] T a3212cbac7024f83976255912a5d23af P 7674f79eb3f2413ea7c46a9ac8d8e639: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:12:16.467280 24560 log.cc:1196] T a3212cbac7024f83976255912a5d23af P 7674f79eb3f2413ea7c46a9ac8d8e639: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288021313853-25311-0/minicluster-data/ts-1/wal/wals/a3212cbac7024f83976255912a5d23af
I20260809 15:12:16.467768 24560 ts_tablet_manager.cc:1939] T a3212cbac7024f83976255912a5d23af P 7674f79eb3f2413ea7c46a9ac8d8e639: Deleting consensus metadata
Aug 09 15:12:16 dist-test-slave-nj21 krb5kdc[23755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.193@KRBTEST.COM
I20260809 15:12:16.469154 23802 catalog_manager.cc:4841] TS 7674f79eb3f2413ea7c46a9ac8d8e639 (127.24.183.194:33751): tablet a3212cbac7024f83976255912a5d23af (table test-table [id=523d8f00b70140afbe6ae3a960bc4a28]) successfully deleted
I20260809 15:12:16.475970 24507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:16.446470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36652 (local address 127.24.183.193:43469)
0809 15:12:16.446686 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:16.446691 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:16.459030 (+ 12339us) server_negotiation.cc:407] Connection header received
0809 15:12:16.459102 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:16.459107 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:16.459199 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:16.459313 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:16.461029 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.462132 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.463982 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.464649 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.464789 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:16.470182 (+ 5393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:16.470193 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:16.470207 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:12:16.470261 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:16.473317 (+ 3056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:16.474117 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:16.474123 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:16.474125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:16.474249 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:16.474606 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:16.474608 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:16.474610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:16.474961 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:16.475201 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:16.475510 (+ 309us) server_negotiation.cc:299] Negotiation successful
0809 15:12:16.475710 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 15:12:16.476646 24097 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:12:16.576264 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 23911
I20260809 15:12:16.599943 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 24111
Aug 09 15:12:16 dist-test-slave-nj21 krb5kdc[23755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.194@KRBTEST.COM
I20260809 15:12:16.614854 24512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:16.600782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41636 (local address 127.24.183.194:33751)
0809 15:12:16.600958 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:16.600962 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:16.600979 (+ 17us) server_negotiation.cc:407] Connection header received
0809 15:12:16.601115 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:16.601119 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:16.601189 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:16.601261 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:16.602823 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.603594 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.605139 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.605557 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.605645 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:16.609590 (+ 3945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:16.609601 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:16.609615 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:12:16.609644 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:16.612223 (+ 2579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:16.612872 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:16.612879 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:16.612881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:16.613020 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:16.613368 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:16.613372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:16.613374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:16.613744 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:16.614036 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:16.614326 (+ 290us) server_negotiation.cc:299] Negotiation successful
0809 15:12:16.614538 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 15:12:16.615499 24288 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:12:16.706988 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 24111
I20260809 15:12:16.729523 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 24299
Aug 09 15:12:16 dist-test-slave-nj21 krb5kdc[23755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.195@KRBTEST.COM
I20260809 15:12:16.743795 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:16.730316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36092 (local address 127.24.183.195:40149)
0809 15:12:16.730502 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:16.730507 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:16.730527 (+ 20us) server_negotiation.cc:407] Connection header received
0809 15:12:16.730668 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:16.730672 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:16.730753 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:16.730855 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:16.732140 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.732796 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.734007 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:16.734462 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:16.734554 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:16.738824 (+ 4270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:16.738846 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:16.738864 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:12:16.738898 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:16.741516 (+ 2618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:16.742113 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:16.742120 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:16.742122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:16.742245 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:16.742528 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:16.742531 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:16.742533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:16.742895 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:16.743107 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:16.743314 (+ 207us) server_negotiation.cc:299] Negotiation successful
0809 15:12:16.743471 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 15:12:16.744447 24478 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:12:16.833397 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 24299
I20260809 15:12:16.859597 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 23774
I20260809 15:12:16.860767 23867 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:12:16.973469 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 23774
2026-08-09T15:12:16Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4500 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T15:12:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:12:17Z Disabled control of system clock
I20260809 15:12:17.063236 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:42657
--webserver_interface=127.24.183.254
--webserver_port=0
--builtin_ntp_servers=127.24.183.212:40329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:42657
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 15:12:17.396876 24574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:17.397146 24574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:17.397236 24574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:17.404384 24574 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:12:17.404485 24574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:17.404529 24574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:17.404585 24574 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:12:17.404628 24574 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:12:17.413573 24574 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:40329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:42657
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:42657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:17.414774 24574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:17.416545 24574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:17.427577 24582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:17.427646 24581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:17.427695 24585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:17.428488 24574 server_base.cc:1034] running on GCE node
I20260809 15:12:17.429211 24574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:17.430519 24574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:17.431733 24574 hybrid_clock.cc:648] HybridClock initialized: now 1786288337431685 us; error 37 us; skew 500 ppm
I20260809 15:12:17.432058 24574 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:12:17.438894 24574 webserver.cc:460] Webserver started at http://127.24.183.254:39911/ using document root <none> and password file <none>
I20260809 15:12:17.439553 24574 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:17.439652 24574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:17.439981 24574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:17.442770 24574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "d6281c225070499e878b61c930a794fe"
format_stamp: "Formatted at 2026-08-09 15:12:17 on dist-test-slave-nj21"
server_key: "f55291980a8360cc73e6fa2767a39bed"
server_key_iv: "b0b9d46d08ffac981d89c298adec344a"
server_key_version: "encryptionkey@0"
I20260809 15:12:17.443557 24574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "d6281c225070499e878b61c930a794fe"
format_stamp: "Formatted at 2026-08-09 15:12:17 on dist-test-slave-nj21"
server_key: "f55291980a8360cc73e6fa2767a39bed"
server_key_iv: "b0b9d46d08ffac981d89c298adec344a"
server_key_version: "encryptionkey@0"
I20260809 15:12:17.448359 24574 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 15:12:17.451521 24594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:17.452740 24574 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260809 15:12:17.452901 24574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "d6281c225070499e878b61c930a794fe"
format_stamp: "Formatted at 2026-08-09 15:12:17 on dist-test-slave-nj21"
server_key: "f55291980a8360cc73e6fa2767a39bed"
server_key_iv: "b0b9d46d08ffac981d89c298adec344a"
server_key_version: "encryptionkey@0"
I20260809 15:12:17.453084 24574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:17.481509 24574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:17.482362 24574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:17.482632 24574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:17.500484 24574 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:42657
I20260809 15:12:17.500504 24675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:42657 every 8 connection(s)
I20260809 15:12:17.501809 24574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:12:17.505441 24676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:17.508579 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 24574
I20260809 15:12:17.508756 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:12:17.509339 25311 external_mini_cluster.cc:1428] Setting key df78bbb220a94ae659ccd00d4d89b1c7
I20260809 15:12:17.517699 24676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6281c225070499e878b61c930a794fe: Bootstrap starting.
I20260809 15:12:17.521133 24676 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6281c225070499e878b61c930a794fe: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:17.522209 24676 log.cc:824] T 00000000000000000000000000000000 P d6281c225070499e878b61c930a794fe: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:17.522758 24681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:17.511335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55000 (local address 127.24.183.254:42657)
0809 15:12:17.512310 (+ 975us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:17.512321 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:17.512364 (+ 43us) server_negotiation.cc:407] Connection header received
0809 15:12:17.513985 (+ 1621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:17.514007 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:17.514477 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:17.514788 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:17.515416 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:17.516744 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:17.518531 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:17.519201 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:17.519324 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:17.520546 (+ 1222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:17.520554 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:17.520566 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:17.521062 (+ 496us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:17.521154 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:17.521174 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:17.521282 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:17.521509 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 15:12:17.521776 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":374,"threads_started":1}
I20260809 15:12:17.525076 24676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6281c225070499e878b61c930a794fe: No bootstrap required, opened a new log
I20260809 15:12:17.532099 24676 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6281c225070499e878b61c930a794fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6281c225070499e878b61c930a794fe" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 42657 } }
I20260809 15:12:17.532413 24676 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6281c225070499e878b61c930a794fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:17.532485 24676 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6281c225070499e878b61c930a794fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6281c225070499e878b61c930a794fe, State: Initialized, Role: FOLLOWER
I20260809 15:12:17.532970 24676 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6281c225070499e878b61c930a794fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6281c225070499e878b61c930a794fe" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 42657 } }
I20260809 15:12:17.533179 24676 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6281c225070499e878b61c930a794fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:12:17.533274 24676 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6281c225070499e878b61c930a794fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:12:17.533406 24676 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6281c225070499e878b61c930a794fe [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:17.535566 24676 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6281c225070499e878b61c930a794fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6281c225070499e878b61c930a794fe" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 42657 } }
I20260809 15:12:17.536041 24676 leader_election.cc:302] T 00000000000000000000000000000000 P d6281c225070499e878b61c930a794fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6281c225070499e878b61c930a794fe; no voters:
I20260809 15:12:17.536542 24676 leader_election.cc:288] T 00000000000000000000000000000000 P d6281c225070499e878b61c930a794fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:12:17.536654 24685 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6281c225070499e878b61c930a794fe [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:12:17.537539 24685 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6281c225070499e878b61c930a794fe [term 1 LEADER]: Becoming Leader. State: Replica: d6281c225070499e878b61c930a794fe, State: Running, Role: LEADER
I20260809 15:12:17.537997 24685 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6281c225070499e878b61c930a794fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6281c225070499e878b61c930a794fe" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 42657 } }
I20260809 15:12:17.538417 24676 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6281c225070499e878b61c930a794fe [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:12:17.540665 24686 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6281c225070499e878b61c930a794fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6281c225070499e878b61c930a794fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6281c225070499e878b61c930a794fe" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 42657 } } }
I20260809 15:12:17.540735 24688 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6281c225070499e878b61c930a794fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6281c225070499e878b61c930a794fe. Latest consensus state: current_term: 1 leader_uuid: "d6281c225070499e878b61c930a794fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6281c225070499e878b61c930a794fe" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 42657 } } }
I20260809 15:12:17.541079 24686 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6281c225070499e878b61c930a794fe [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:17.541083 24688 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6281c225070499e878b61c930a794fe [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:17.542688 24696 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:12:17.546926 24696 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:12:17.554595 24696 catalog_manager.cc:1347] Generated new cluster ID: aefc7dbfee7d4aaf8bb92eecc0b2d099
I20260809 15:12:17.554702 24696 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:12:17.565862 24696 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:12:17.567564 24696 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:12:17.582762 24696 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6281c225070499e878b61c930a794fe: Generated new TSK 0
I20260809 15:12:17.583734 24696 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:12:17.595086 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.193:0
--local_ip_for_outbound_sockets=127.24.183.193
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:42657
--builtin_ntp_servers=127.24.183.212:40329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 15:12:17.936486 24714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:17.936753 24714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:17.936873 24714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:17.944168 24714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:17.944262 24714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:17.944392 24714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.193
I20260809 15:12:17.953351 24714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:40329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.183.193
--webserver_port=0
--tserver_master_addrs=127.24.183.254:42657
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.193
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:17.954602 24714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:17.956445 24714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:17.969643 24721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:17.969738 24722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:17.969704 24725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:17.970752 24714 server_base.cc:1034] running on GCE node
I20260809 15:12:17.971313 24714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:17.972224 24714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:17.973433 24714 hybrid_clock.cc:648] HybridClock initialized: now 1786288337973390 us; error 23 us; skew 500 ppm
I20260809 15:12:17.973713 24714 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:12:17.975874 24714 webserver.cc:460] Webserver started at http://127.24.183.193:33199/ using document root <none> and password file <none>
I20260809 15:12:17.976522 24714 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:17.976624 24714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:17.976925 24714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:17.979703 24714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-0/data/instance:
uuid: "2e1c099e32914df38df95a77b5ecc220"
format_stamp: "Formatted at 2026-08-09 15:12:17 on dist-test-slave-nj21"
server_key: "b9e0742c18079e57eaace492d172043d"
server_key_iv: "173ed8dad6ddfa90c057024e5aacb61d"
server_key_version: "encryptionkey@0"
I20260809 15:12:17.980482 24714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance:
uuid: "2e1c099e32914df38df95a77b5ecc220"
format_stamp: "Formatted at 2026-08-09 15:12:17 on dist-test-slave-nj21"
server_key: "b9e0742c18079e57eaace492d172043d"
server_key_iv: "173ed8dad6ddfa90c057024e5aacb61d"
server_key_version: "encryptionkey@0"
I20260809 15:12:17.985222 24714 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 15:12:17.988464 24734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:17.989703 24714 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:12:17.989876 24714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-0/wal
uuid: "2e1c099e32914df38df95a77b5ecc220"
format_stamp: "Formatted at 2026-08-09 15:12:17 on dist-test-slave-nj21"
server_key: "b9e0742c18079e57eaace492d172043d"
server_key_iv: "173ed8dad6ddfa90c057024e5aacb61d"
server_key_version: "encryptionkey@0"
I20260809 15:12:17.990067 24714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:18.007810 24714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:18.008524 24714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:18.008821 24714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:18.009691 24714 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:12:18.011062 24714 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:12:18.011149 24714 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:18.011226 24714 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:12:18.011281 24714 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:18.042126 24714 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.193:45119
I20260809 15:12:18.042145 24909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.193:45119 every 8 connection(s)
I20260809 15:12:18.043475 24714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-0/data/info.pb
I20260809 15:12:18.049379 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 24714
I20260809 15:12:18.049566 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-0/wal/instance
I20260809 15:12:18.050109 25311 external_mini_cluster.cc:1428] Setting key 93ca5e06322db47dc086ceb8fb582e17
I20260809 15:12:18.053035 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.194:0
--local_ip_for_outbound_sockets=127.24.183.194
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:42657
--builtin_ntp_servers=127.24.183.212:40329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 15:12:18.056460 24915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:18.045821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.193:41739 (local address 127.24.183.254:42657)
0809 15:12:18.046260 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:18.046267 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:18.047428 (+ 1161us) server_negotiation.cc:407] Connection header received
0809 15:12:18.049508 (+ 2080us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:18.049515 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:18.049610 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:18.049730 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:18.050783 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:18.051536 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:18.053688 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:18.054115 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:18.054216 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:18.055726 (+ 1510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:18.055733 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:18.055736 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:18.055754 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:18.055795 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:18.055808 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:18.055890 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:18.056108 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 15:12:18.056186 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":199,"threads_started":1}
I20260809 15:12:18.057722 24916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:18.046397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:42657 (local address 127.24.183.193:41739)
0809 15:12:18.047253 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:18.047320 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:18.049221 (+ 1901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:18.049904 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:18.049912 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:18.049994 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:12:18.050598 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:18.050623 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:18.051716 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:18.051722 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:18.053548 (+ 1826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:18.053562 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:18.054335 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:18.054343 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:18.054607 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:18.055376 (+ 769us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:12:18.055380 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:12:18.055427 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:12:18.055522 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:12:18.055916 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:12:18.055926 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:12:18.055934 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 15:12:18.056093 (+ 159us) client_negotiation.cc:240] Negotiation successful
0809 15:12:18.056478 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":252,"threads_started":1}
I20260809 15:12:18.059108 24910 heartbeater.cc:347] Connected to a master server at 127.24.183.254:42657
I20260809 15:12:18.059407 24910 heartbeater.cc:464] Registering TS with master...
I20260809 15:12:18.060292 24910 heartbeater.cc:511] Master 127.24.183.254:42657 requested a full tablet report, sending...
I20260809 15:12:18.062268 24621 ts_manager.cc:194] Registered new tserver with Master: 2e1c099e32914df38df95a77b5ecc220 (127.24.183.193:45119)
I20260809 15:12:18.063783 24621 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.183.193:41739
W20260809 15:12:18.384742 24917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:18.384999 24917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:18.385123 24917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:18.392347 24917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:18.392448 24917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:18.392591 24917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.194
I20260809 15:12:18.401464 24917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:40329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.183.194
--webserver_port=0
--tserver_master_addrs=127.24.183.254:42657
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.194
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:18.402727 24917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:18.404560 24917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:18.417040 24931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:18.417126 24926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:18.417130 24927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:18.418087 24917 server_base.cc:1034] running on GCE node
I20260809 15:12:18.418624 24917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:18.419445 24917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:18.420630 24917 hybrid_clock.cc:648] HybridClock initialized: now 1786288338420565 us; error 52 us; skew 500 ppm
I20260809 15:12:18.420897 24917 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:12:18.422923 24917 webserver.cc:460] Webserver started at http://127.24.183.194:33155/ using document root <none> and password file <none>
I20260809 15:12:18.423533 24917 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:18.423619 24917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:18.423926 24917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:18.426743 24917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-1/data/instance:
uuid: "436a238b0f8d42d2ab936da91e19e127"
format_stamp: "Formatted at 2026-08-09 15:12:18 on dist-test-slave-nj21"
server_key: "3a3171d4d07151da4d9a84b8a024a2e9"
server_key_iv: "cd7843cc17fd746bcee65b7ef08a5713"
server_key_version: "encryptionkey@0"
I20260809 15:12:18.427462 24917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance:
uuid: "436a238b0f8d42d2ab936da91e19e127"
format_stamp: "Formatted at 2026-08-09 15:12:18 on dist-test-slave-nj21"
server_key: "3a3171d4d07151da4d9a84b8a024a2e9"
server_key_iv: "cd7843cc17fd746bcee65b7ef08a5713"
server_key_version: "encryptionkey@0"
I20260809 15:12:18.432199 24917 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 15:12:18.435309 24937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:18.436561 24917 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:12:18.436707 24917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-1/wal
uuid: "436a238b0f8d42d2ab936da91e19e127"
format_stamp: "Formatted at 2026-08-09 15:12:18 on dist-test-slave-nj21"
server_key: "3a3171d4d07151da4d9a84b8a024a2e9"
server_key_iv: "cd7843cc17fd746bcee65b7ef08a5713"
server_key_version: "encryptionkey@0"
I20260809 15:12:18.436892 24917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:18.464540 24917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:18.465349 24917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:18.465631 24917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:18.466588 24917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:12:18.468044 24917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:12:18.468142 24917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:18.468256 24917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:12:18.468326 24917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:18.498322 24917 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.194:45769
I20260809 15:12:18.498392 25102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.194:45769 every 8 connection(s)
I20260809 15:12:18.499599 24917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-1/data/info.pb
I20260809 15:12:18.503186 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 24917
I20260809 15:12:18.503388 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-1/wal/instance
I20260809 15:12:18.503962 25311 external_mini_cluster.cc:1428] Setting key 101b5bfefa5b7bf067b0ae928a0e88c3
I20260809 15:12:18.507438 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.183.195:0
--local_ip_for_outbound_sockets=127.24.183.195
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:42657
--builtin_ntp_servers=127.24.183.212:40329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 15:12:18.514250 24915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:18.502465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.194:37535 (local address 127.24.183.254:42657)
0809 15:12:18.502664 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:18.502671 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:18.505868 (+ 3197us) server_negotiation.cc:407] Connection header received
0809 15:12:18.508115 (+ 2247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:18.508120 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:18.508207 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:18.508298 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:18.509373 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:18.510128 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:18.511626 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:18.512106 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:18.512216 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:18.513469 (+ 1253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:18.513476 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:18.513481 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:18.513498 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:18.513529 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:18.513540 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:18.513612 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:18.513822 (+ 210us) server_negotiation.cc:299] Negotiation successful
0809 15:12:18.513929 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 15:12:18.515312 25107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:18.503203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:42657 (local address 127.24.183.194:37535)
0809 15:12:18.505713 (+ 2510us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:18.505765 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:18.507866 (+ 2101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:18.508563 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:18.508575 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:18.508665 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:12:18.509209 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:18.509229 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:18.510276 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:18.510280 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:18.511487 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:18.511499 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:18.512318 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:18.512327 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:18.512539 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:18.513172 (+ 633us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:12:18.513175 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:12:18.513218 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:12:18.513289 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:12:18.513623 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:12:18.513634 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:12:18.513642 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 15:12:18.513797 (+ 155us) client_negotiation.cc:240] Negotiation successful
0809 15:12:18.514162 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1845,"spinlock_wait_cycles":40704,"thread_start_us":1741,"threads_started":1}
I20260809 15:12:18.516615 25103 heartbeater.cc:347] Connected to a master server at 127.24.183.254:42657
I20260809 15:12:18.516907 25103 heartbeater.cc:464] Registering TS with master...
I20260809 15:12:18.517642 25103 heartbeater.cc:511] Master 127.24.183.254:42657 requested a full tablet report, sending...
I20260809 15:12:18.518970 24621 ts_manager.cc:194] Registered new tserver with Master: 436a238b0f8d42d2ab936da91e19e127 (127.24.183.194:45769)
I20260809 15:12:18.519929 24621 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.183.194:37535
W20260809 15:12:18.841694 25109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:18.841998 25109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:18.842126 25109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:18.849500 25109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:18.849610 25109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:18.849768 25109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.183.195
I20260809 15:12:18.859036 25109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:40329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.183.195
--webserver_port=0
--tserver_master_addrs=127.24.183.254:42657
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.183.195
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:18.860414 25109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:18.862247 25109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:18.875092 25115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:18.875129 25116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:18.875032 25122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:18.876196 25109 server_base.cc:1034] running on GCE node
I20260809 15:12:18.876830 25109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:18.877782 25109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:18.879009 25109 hybrid_clock.cc:648] HybridClock initialized: now 1786288338878951 us; error 49 us; skew 500 ppm
I20260809 15:12:18.879324 25109 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:12:18.881495 25109 webserver.cc:460] Webserver started at http://127.24.183.195:35187/ using document root <none> and password file <none>
I20260809 15:12:18.882153 25109 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:18.882246 25109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:18.882570 25109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:18.885373 25109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-2/data/instance:
uuid: "a9683a2b3ae742c5b93beca3cfdf94bd"
format_stamp: "Formatted at 2026-08-09 15:12:18 on dist-test-slave-nj21"
server_key: "e3e07cb1ba1c0494c34ed005157d45d8"
server_key_iv: "6b5868814f0b53e29f6e22e985898beb"
server_key_version: "encryptionkey@0"
I20260809 15:12:18.886139 25109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance:
uuid: "a9683a2b3ae742c5b93beca3cfdf94bd"
format_stamp: "Formatted at 2026-08-09 15:12:18 on dist-test-slave-nj21"
server_key: "e3e07cb1ba1c0494c34ed005157d45d8"
server_key_iv: "6b5868814f0b53e29f6e22e985898beb"
server_key_version: "encryptionkey@0"
I20260809 15:12:18.891078 25109 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 15:12:18.894486 25128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:18.895783 25109 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:12:18.895926 25109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-2/wal
uuid: "a9683a2b3ae742c5b93beca3cfdf94bd"
format_stamp: "Formatted at 2026-08-09 15:12:18 on dist-test-slave-nj21"
server_key: "e3e07cb1ba1c0494c34ed005157d45d8"
server_key_iv: "6b5868814f0b53e29f6e22e985898beb"
server_key_version: "encryptionkey@0"
I20260809 15:12:18.896130 25109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:18.924015 25109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:18.924787 25109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:18.925063 25109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:18.926020 25109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:12:18.927484 25109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:12:18.927575 25109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:18.927707 25109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:12:18.927780 25109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260809 15:12:18.958297 25109 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.195:43267
I20260809 15:12:18.958381 25295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.195:43267 every 8 connection(s)
I20260809 15:12:18.959554 25109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-2/data/info.pb
I20260809 15:12:18.967904 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 25109
I20260809 15:12:18.968060 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-2/wal/instance
I20260809 15:12:18.968467 25311 external_mini_cluster.cc:1428] Setting key c9ca569b90362ebee964fa2f3f576ff2
I20260809 15:12:18.971737 24915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:18.961979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:36227 (local address 127.24.183.254:42657)
0809 15:12:18.962201 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:18.962207 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:18.963517 (+ 1310us) server_negotiation.cc:407] Connection header received
0809 15:12:18.965281 (+ 1764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:18.965285 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:18.965357 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:18.965445 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:18.966539 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:18.967277 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:18.968858 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:18.969412 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:18.969531 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:18.970916 (+ 1385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:18.970921 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:18.970924 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:18.970951 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:18.970985 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:18.970997 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:18.971072 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:18.971310 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 15:12:18.971417 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 15:12:18.972654 25298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:18.962488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:42657 (local address 127.24.183.195:36227)
0809 15:12:18.963372 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:18.963407 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:18.965060 (+ 1653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:18.965623 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:18.965631 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:18.965711 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:12:18.966404 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:18.966422 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:18.967380 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:18.967383 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:18.968720 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:18.968732 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:18.969637 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:18.969646 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:18.969878 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:18.970585 (+ 707us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:12:18.970589 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:12:18.970638 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:12:18.970730 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:12:18.971092 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:12:18.971103 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:12:18.971112 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 15:12:18.971287 (+ 175us) client_negotiation.cc:240] Negotiation successful
0809 15:12:18.971583 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":295,"threads_started":1}
I20260809 15:12:18.973784 25296 heartbeater.cc:347] Connected to a master server at 127.24.183.254:42657
I20260809 15:12:18.974040 25296 heartbeater.cc:464] Registering TS with master...
I20260809 15:12:18.974735 25296 heartbeater.cc:511] Master 127.24.183.254:42657 requested a full tablet report, sending...
I20260809 15:12:18.976023 24621 ts_manager.cc:194] Registered new tserver with Master: a9683a2b3ae742c5b93beca3cfdf94bd (127.24.183.195:43267)
I20260809 15:12:18.976780 24621 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.183.195:36227
I20260809 15:12:18.983258 25311 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:12:18.992493 24915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:18.987202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55010 (local address 127.24.183.254:42657)
0809 15:12:18.987363 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:18.987369 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:18.987821 (+ 452us) server_negotiation.cc:407] Connection header received
0809 15:12:18.987951 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:18.987954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:18.988027 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:18.988103 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:18.988510 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:18.989208 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:18.990544 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:18.990938 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:18.991021 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:18.991966 (+ 945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:18.991971 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:18.991974 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:18.991990 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:18.992020 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:18.992031 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:18.992095 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:18.992177 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 15:12:18.992246 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 15:12:18.995978 24621 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55010:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:12:19.002779 24621 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:12:19.040179 25316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:19.013971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.195:43267 (local address 127.0.0.1:47994)
0809 15:12:19.016031 (+ 2060us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:19.016080 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:19.016250 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:19.032691 (+ 16441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:19.032699 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:19.032733 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:12:19.033163 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:19.033183 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:19.035483 (+ 2300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:19.035489 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:19.037557 (+ 2068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:19.037570 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:19.039272 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:19.039279 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:19.039471 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:19.039520 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 15:12:19.039655 (+ 135us) client_negotiation.cc:240] Negotiation successful
0809 15:12:19.039864 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1788,"thread_start_us":2842,"threads_started":1}
I20260809 15:12:19.040290 25317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:19.014135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47994 (local address 127.24.183.195:43267)
0809 15:12:19.031630 (+ 17495us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:19.031650 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:19.031710 (+ 60us) server_negotiation.cc:407] Connection header received
0809 15:12:19.031834 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:19.031843 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:19.032118 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:19.033388 (+ 1270us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:12:19.033693 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:19.035313 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:19.037745 (+ 2432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:19.039107 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:19.039286 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:19.039525 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:19.039718 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 15:12:19.039985 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17289,"thread_start_us":17101,"threads_started":1}
I20260809 15:12:19.042244 25321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:19.017827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36816 (local address 127.24.183.194:45769)
0809 15:12:19.033913 (+ 16086us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:19.033919 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:19.033940 (+ 21us) server_negotiation.cc:407] Connection header received
0809 15:12:19.034013 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:19.034018 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:19.034163 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:19.034326 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:12:19.035158 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:19.036632 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:19.039128 (+ 2496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:19.040503 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:19.040638 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:19.040812 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:19.041774 (+ 962us) server_negotiation.cc:299] Negotiation successful
0809 15:12:19.041939 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14351,"thread_start_us":14228,"threads_started":1}
I20260809 15:12:19.042500 25319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:19.015205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50786 (local address 127.24.183.193:45119)
0809 15:12:19.031899 (+ 16694us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:19.031910 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:19.032968 (+ 1058us) server_negotiation.cc:407] Connection header received
0809 15:12:19.033089 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:19.033097 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:19.033325 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:19.033507 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:12:19.034287 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:19.035778 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:19.038374 (+ 2596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:19.039889 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:19.040054 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:19.040244 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:19.041948 (+ 1704us) server_negotiation.cc:299] Negotiation successful
0809 15:12:19.042110 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16554,"thread_start_us":16466,"threads_started":1}
I20260809 15:12:19.042724 25318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:19.017591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:45769 (local address 127.0.0.1:36816)
0809 15:12:19.033628 (+ 16037us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:19.033642 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:19.033725 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:19.034575 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:19.034579 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:19.034605 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:12:19.034980 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:19.034993 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:19.036795 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:19.036803 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:19.038927 (+ 2124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:19.038942 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:19.040921 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:19.040926 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:19.041202 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:19.041311 (+ 109us) client_negotiation.cc:769] Sending connection context
0809 15:12:19.041407 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 15:12:19.041548 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15936,"mutex_wait_us":91,"thread_start_us":14817,"threads_started":1}
I20260809 15:12:19.042883 25320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:19.014564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:45119 (local address 127.0.0.1:50786)
0809 15:12:19.032691 (+ 18127us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:19.032711 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:19.032821 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:19.033731 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:19.033735 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:19.033762 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:12:19.034089 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:19.034101 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:19.035953 (+ 1852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:19.035958 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:19.038158 (+ 2200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:19.038173 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:19.040918 (+ 2745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:19.040926 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:19.041209 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:19.041311 (+ 102us) client_negotiation.cc:769] Sending connection context
0809 15:12:19.041395 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 15:12:19.041522 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18003,"mutex_wait_us":86,"thread_start_us":18839,"threads_started":1}
I20260809 15:12:19.045603 25202 tablet_service.cc:1467] Processing CreateTablet for tablet 4087f96a425841b182a2206ca764a850 (DEFAULT_TABLE table=test-table [id=9d50692bb7aa45dda3aefc451e425e94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:12:19.046471 24810 tablet_service.cc:1467] Processing CreateTablet for tablet 4087f96a425841b182a2206ca764a850 (DEFAULT_TABLE table=test-table [id=9d50692bb7aa45dda3aefc451e425e94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:12:19.047348 25202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4087f96a425841b182a2206ca764a850. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:19.046988 25008 tablet_service.cc:1467] Processing CreateTablet for tablet 4087f96a425841b182a2206ca764a850 (DEFAULT_TABLE table=test-table [id=9d50692bb7aa45dda3aefc451e425e94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:12:19.047969 24810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4087f96a425841b182a2206ca764a850. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:19.048281 25008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4087f96a425841b182a2206ca764a850. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:19.058996 25323 tablet_bootstrap.cc:492] T 4087f96a425841b182a2206ca764a850 P a9683a2b3ae742c5b93beca3cfdf94bd: Bootstrap starting.
I20260809 15:12:19.062254 25323 tablet_bootstrap.cc:654] T 4087f96a425841b182a2206ca764a850 P a9683a2b3ae742c5b93beca3cfdf94bd: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:19.062255 25324 tablet_bootstrap.cc:492] T 4087f96a425841b182a2206ca764a850 P 2e1c099e32914df38df95a77b5ecc220: Bootstrap starting.
I20260809 15:12:19.063620 25323 log.cc:824] T 4087f96a425841b182a2206ca764a850 P a9683a2b3ae742c5b93beca3cfdf94bd: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:19.064342 25326 tablet_bootstrap.cc:492] T 4087f96a425841b182a2206ca764a850 P 436a238b0f8d42d2ab936da91e19e127: Bootstrap starting.
I20260809 15:12:19.066051 25324 tablet_bootstrap.cc:654] T 4087f96a425841b182a2206ca764a850 P 2e1c099e32914df38df95a77b5ecc220: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:19.066830 24910 heartbeater.cc:503] Master 127.24.183.254:42657 was elected leader, sending a full tablet report...
I20260809 15:12:19.067358 25324 log.cc:824] T 4087f96a425841b182a2206ca764a850 P 2e1c099e32914df38df95a77b5ecc220: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:19.067446 25326 tablet_bootstrap.cc:654] T 4087f96a425841b182a2206ca764a850 P 436a238b0f8d42d2ab936da91e19e127: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:19.068648 25326 log.cc:824] T 4087f96a425841b182a2206ca764a850 P 436a238b0f8d42d2ab936da91e19e127: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:19.073892 25324 tablet_bootstrap.cc:492] T 4087f96a425841b182a2206ca764a850 P 2e1c099e32914df38df95a77b5ecc220: No bootstrap required, opened a new log
I20260809 15:12:19.074044 25323 tablet_bootstrap.cc:492] T 4087f96a425841b182a2206ca764a850 P a9683a2b3ae742c5b93beca3cfdf94bd: No bootstrap required, opened a new log
I20260809 15:12:19.074048 25326 tablet_bootstrap.cc:492] T 4087f96a425841b182a2206ca764a850 P 436a238b0f8d42d2ab936da91e19e127: No bootstrap required, opened a new log
I20260809 15:12:19.074147 25324 ts_tablet_manager.cc:1397] T 4087f96a425841b182a2206ca764a850 P 2e1c099e32914df38df95a77b5ecc220: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.005s
I20260809 15:12:19.074316 25326 ts_tablet_manager.cc:1397] T 4087f96a425841b182a2206ca764a850 P 436a238b0f8d42d2ab936da91e19e127: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260809 15:12:19.074311 25323 ts_tablet_manager.cc:1397] T 4087f96a425841b182a2206ca764a850 P a9683a2b3ae742c5b93beca3cfdf94bd: Time spent bootstrapping tablet: real 0.016s user 0.011s sys 0.000s
I20260809 15:12:19.081351 25324 raft_consensus.cc:348] T 4087f96a425841b182a2206ca764a850 P 2e1c099e32914df38df95a77b5ecc220 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9683a2b3ae742c5b93beca3cfdf94bd" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 43267 } } peers { permanent_uuid: "436a238b0f8d42d2ab936da91e19e127" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 45769 } } peers { permanent_uuid: "2e1c099e32914df38df95a77b5ecc220" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 45119 } }
I20260809 15:12:19.081751 25324 raft_consensus.cc:374] T 4087f96a425841b182a2206ca764a850 P 2e1c099e32914df38df95a77b5ecc220 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:19.081846 25324 raft_consensus.cc:729] T 4087f96a425841b182a2206ca764a850 P 2e1c099e32914df38df95a77b5ecc220 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e1c099e32914df38df95a77b5ecc220, State: Initialized, Role: FOLLOWER
I20260809 15:12:19.082402 25324 consensus_queue.cc:260] T 4087f96a425841b182a2206ca764a850 P 2e1c099e32914df38df95a77b5ecc220 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9683a2b3ae742c5b93beca3cfdf94bd" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 43267 } } peers { permanent_uuid: "436a238b0f8d42d2ab936da91e19e127" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 45769 } } peers { permanent_uuid: "2e1c099e32914df38df95a77b5ecc220" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 45119 } }
I20260809 15:12:19.084111 25324 ts_tablet_manager.cc:1428] T 4087f96a425841b182a2206ca764a850 P 2e1c099e32914df38df95a77b5ecc220: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260809 15:12:19.084604 25323 raft_consensus.cc:348] T 4087f96a425841b182a2206ca764a850 P a9683a2b3ae742c5b93beca3cfdf94bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9683a2b3ae742c5b93beca3cfdf94bd" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 43267 } } peers { permanent_uuid: "436a238b0f8d42d2ab936da91e19e127" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 45769 } } peers { permanent_uuid: "2e1c099e32914df38df95a77b5ecc220" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 45119 } }
I20260809 15:12:19.085027 25323 raft_consensus.cc:374] T 4087f96a425841b182a2206ca764a850 P a9683a2b3ae742c5b93beca3cfdf94bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:19.084820 25326 raft_consensus.cc:348] T 4087f96a425841b182a2206ca764a850 P 436a238b0f8d42d2ab936da91e19e127 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9683a2b3ae742c5b93beca3cfdf94bd" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 43267 } } peers { permanent_uuid: "436a238b0f8d42d2ab936da91e19e127" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 45769 } } peers { permanent_uuid: "2e1c099e32914df38df95a77b5ecc220" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 45119 } }
I20260809 15:12:19.085225 25326 raft_consensus.cc:374] T 4087f96a425841b182a2206ca764a850 P 436a238b0f8d42d2ab936da91e19e127 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:19.085225 25323 raft_consensus.cc:729] T 4087f96a425841b182a2206ca764a850 P a9683a2b3ae742c5b93beca3cfdf94bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9683a2b3ae742c5b93beca3cfdf94bd, State: Initialized, Role: FOLLOWER
I20260809 15:12:19.085331 25326 raft_consensus.cc:729] T 4087f96a425841b182a2206ca764a850 P 436a238b0f8d42d2ab936da91e19e127 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 436a238b0f8d42d2ab936da91e19e127, State: Initialized, Role: FOLLOWER
I20260809 15:12:19.085892 25323 consensus_queue.cc:260] T 4087f96a425841b182a2206ca764a850 P a9683a2b3ae742c5b93beca3cfdf94bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9683a2b3ae742c5b93beca3cfdf94bd" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 43267 } } peers { permanent_uuid: "436a238b0f8d42d2ab936da91e19e127" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 45769 } } peers { permanent_uuid: "2e1c099e32914df38df95a77b5ecc220" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 45119 } }
I20260809 15:12:19.085883 25326 consensus_queue.cc:260] T 4087f96a425841b182a2206ca764a850 P 436a238b0f8d42d2ab936da91e19e127 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9683a2b3ae742c5b93beca3cfdf94bd" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 43267 } } peers { permanent_uuid: "436a238b0f8d42d2ab936da91e19e127" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 45769 } } peers { permanent_uuid: "2e1c099e32914df38df95a77b5ecc220" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 45119 } }
I20260809 15:12:19.087038 25103 heartbeater.cc:503] Master 127.24.183.254:42657 was elected leader, sending a full tablet report...
I20260809 15:12:19.087430 25296 heartbeater.cc:503] Master 127.24.183.254:42657 was elected leader, sending a full tablet report...
I20260809 15:12:19.087584 25326 ts_tablet_manager.cc:1428] T 4087f96a425841b182a2206ca764a850 P 436a238b0f8d42d2ab936da91e19e127: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260809 15:12:19.088014 25323 ts_tablet_manager.cc:1428] T 4087f96a425841b182a2206ca764a850 P a9683a2b3ae742c5b93beca3cfdf94bd: Time spent starting tablet: real 0.014s user 0.015s sys 0.001s
W20260809 15:12:19.211179 25297 tablet.cc:2369] T 4087f96a425841b182a2206ca764a850 P a9683a2b3ae742c5b93beca3cfdf94bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:12:19.257300 25106 tablet.cc:2369] T 4087f96a425841b182a2206ca764a850 P 436a238b0f8d42d2ab936da91e19e127: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:12:19.302162 24913 tablet.cc:2369] T 4087f96a425841b182a2206ca764a850 P 2e1c099e32914df38df95a77b5ecc220: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:12:19.318640 25333 raft_consensus.cc:482] T 4087f96a425841b182a2206ca764a850 P a9683a2b3ae742c5b93beca3cfdf94bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:12:19.318941 25333 raft_consensus.cc:504] T 4087f96a425841b182a2206ca764a850 P a9683a2b3ae742c5b93beca3cfdf94bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9683a2b3ae742c5b93beca3cfdf94bd" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 43267 } } peers { permanent_uuid: "436a238b0f8d42d2ab936da91e19e127" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 45769 } } peers { permanent_uuid: "2e1c099e32914df38df95a77b5ecc220" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 45119 } }
I20260809 15:12:19.320701 25333 leader_election.cc:288] T 4087f96a425841b182a2206ca764a850 P a9683a2b3ae742c5b93beca3cfdf94bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 436a238b0f8d42d2ab936da91e19e127 (127.24.183.194:45769), 2e1c099e32914df38df95a77b5ecc220 (127.24.183.193:45119)
I20260809 15:12:19.327142 25319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:19.321215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:36667 (local address 127.24.183.193:45119)
0809 15:12:19.321400 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:19.321407 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:19.321428 (+ 21us) server_negotiation.cc:407] Connection header received
0809 15:12:19.321647 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:19.321651 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:19.321718 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:19.321813 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:12:19.322339 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:19.323303 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:19.325237 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:19.326306 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:19.326418 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:19.326467 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:19.326849 (+ 382us) server_negotiation.cc:299] Negotiation successful
0809 15:12:19.326938 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 15:12:19.327262 25298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:19.321130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:45119 (local address 127.24.183.195:36667)
0809 15:12:19.321341 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:19.321363 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:19.321494 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:19.321922 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:19.321928 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:19.321969 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:12:19.322173 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:19.322182 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:19.323433 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:19.323439 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:19.325080 (+ 1641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:19.325092 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:19.326469 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:19.326475 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:19.326648 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:19.326684 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 15:12:19.326827 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 15:12:19.326968 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 15:12:19.327950 25334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:19.321475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.194:45769 (local address 127.24.183.195:35025)
0809 15:12:19.322172 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:19.322187 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:19.322294 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:19.322602 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:19.322607 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:19.322633 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:12:19.322991 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:19.323002 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:19.324247 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:19.324251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:19.325856 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:19.325871 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:19.327278 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:19.327284 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:19.327448 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:19.327480 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 15:12:19.327574 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 15:12:19.327697 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"mutex_wait_us":3,"thread_start_us":396,"threads_started":1}
I20260809 15:12:19.328055 25321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:19.321288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.183.195:35025 (local address 127.24.183.194:45769)
0809 15:12:19.321493 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:19.321500 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:19.322272 (+ 772us) server_negotiation.cc:407] Connection header received
0809 15:12:19.322436 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:19.322440 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:19.322501 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:19.322569 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:12:19.323127 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:19.324099 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:19.326072 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:19.327095 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:19.327258 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:19.327326 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:19.327628 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 15:12:19.327797 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 15:12:19.328457 24838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4087f96a425841b182a2206ca764a850" candidate_uuid: "a9683a2b3ae742c5b93beca3cfdf94bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e1c099e32914df38df95a77b5ecc220" is_pre_election: true
I20260809 15:12:19.329048 24838 raft_consensus.cc:2454] T 4087f96a425841b182a2206ca764a850 P 2e1c099e32914df38df95a77b5ecc220 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9683a2b3ae742c5b93beca3cfdf94bd in term 0.
I20260809 15:12:19.329110 25037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4087f96a425841b182a2206ca764a850" candidate_uuid: "a9683a2b3ae742c5b93beca3cfdf94bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "436a238b0f8d42d2ab936da91e19e127" is_pre_election: true
I20260809 15:12:19.329681 25037 raft_consensus.cc:2454] T 4087f96a425841b182a2206ca764a850 P 436a238b0f8d42d2ab936da91e19e127 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9683a2b3ae742c5b93beca3cfdf94bd in term 0.
I20260809 15:12:19.329818 25132 leader_election.cc:302] T 4087f96a425841b182a2206ca764a850 P a9683a2b3ae742c5b93beca3cfdf94bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e1c099e32914df38df95a77b5ecc220, a9683a2b3ae742c5b93beca3cfdf94bd; no voters:
I20260809 15:12:19.330224 25333 raft_consensus.cc:2781] T 4087f96a425841b182a2206ca764a850 P a9683a2b3ae742c5b93beca3cfdf94bd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:12:19.330351 25333 raft_consensus.cc:482] T 4087f96a425841b182a2206ca764a850 P a9683a2b3ae742c5b93beca3cfdf94bd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:12:19.330451 25333 raft_consensus.cc:3037] T 4087f96a425841b182a2206ca764a850 P a9683a2b3ae742c5b93beca3cfdf94bd [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:19.332691 25333 raft_consensus.cc:504] T 4087f96a425841b182a2206ca764a850 P a9683a2b3ae742c5b93beca3cfdf94bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9683a2b3ae742c5b93beca3cfdf94bd" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 43267 } } peers { permanent_uuid: "436a238b0f8d42d2ab936da91e19e127" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 45769 } } peers { permanent_uuid: "2e1c099e32914df38df95a77b5ecc220" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 45119 } }
I20260809 15:12:19.333256 25333 leader_election.cc:288] T 4087f96a425841b182a2206ca764a850 P a9683a2b3ae742c5b93beca3cfdf94bd [CANDIDATE]: Term 1 election: Requested vote from peers 436a238b0f8d42d2ab936da91e19e127 (127.24.183.194:45769), 2e1c099e32914df38df95a77b5ecc220 (127.24.183.193:45119)
I20260809 15:12:19.333634 25037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4087f96a425841b182a2206ca764a850" candidate_uuid: "a9683a2b3ae742c5b93beca3cfdf94bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "436a238b0f8d42d2ab936da91e19e127"
I20260809 15:12:19.333719 24838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4087f96a425841b182a2206ca764a850" candidate_uuid: "a9683a2b3ae742c5b93beca3cfdf94bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e1c099e32914df38df95a77b5ecc220"
I20260809 15:12:19.333793 25037 raft_consensus.cc:3037] T 4087f96a425841b182a2206ca764a850 P 436a238b0f8d42d2ab936da91e19e127 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:19.333912 24838 raft_consensus.cc:3037] T 4087f96a425841b182a2206ca764a850 P 2e1c099e32914df38df95a77b5ecc220 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:19.336025 25037 raft_consensus.cc:2454] T 4087f96a425841b182a2206ca764a850 P 436a238b0f8d42d2ab936da91e19e127 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9683a2b3ae742c5b93beca3cfdf94bd in term 1.
I20260809 15:12:19.336246 24838 raft_consensus.cc:2454] T 4087f96a425841b182a2206ca764a850 P 2e1c099e32914df38df95a77b5ecc220 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9683a2b3ae742c5b93beca3cfdf94bd in term 1.
I20260809 15:12:19.336400 25133 leader_election.cc:302] T 4087f96a425841b182a2206ca764a850 P a9683a2b3ae742c5b93beca3cfdf94bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 436a238b0f8d42d2ab936da91e19e127, a9683a2b3ae742c5b93beca3cfdf94bd; no voters:
I20260809 15:12:19.336692 25333 raft_consensus.cc:2781] T 4087f96a425841b182a2206ca764a850 P a9683a2b3ae742c5b93beca3cfdf94bd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:12:19.337108 25333 raft_consensus.cc:686] T 4087f96a425841b182a2206ca764a850 P a9683a2b3ae742c5b93beca3cfdf94bd [term 1 LEADER]: Becoming Leader. State: Replica: a9683a2b3ae742c5b93beca3cfdf94bd, State: Running, Role: LEADER
I20260809 15:12:19.337531 25333 consensus_queue.cc:237] T 4087f96a425841b182a2206ca764a850 P a9683a2b3ae742c5b93beca3cfdf94bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9683a2b3ae742c5b93beca3cfdf94bd" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 43267 } } peers { permanent_uuid: "436a238b0f8d42d2ab936da91e19e127" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 45769 } } peers { permanent_uuid: "2e1c099e32914df38df95a77b5ecc220" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 45119 } }
I20260809 15:12:19.341956 24621 catalog_manager.cc:5495] T 4087f96a425841b182a2206ca764a850 P a9683a2b3ae742c5b93beca3cfdf94bd reported cstate change: term changed from 0 to 1, leader changed from <none> to a9683a2b3ae742c5b93beca3cfdf94bd (127.24.183.195). New cstate: current_term: 1 leader_uuid: "a9683a2b3ae742c5b93beca3cfdf94bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9683a2b3ae742c5b93beca3cfdf94bd" member_type: VOTER last_known_addr { host: "127.24.183.195" port: 43267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "436a238b0f8d42d2ab936da91e19e127" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 45769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e1c099e32914df38df95a77b5ecc220" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 45119 } health_report { overall_health: UNKNOWN } } }
I20260809 15:12:19.391039 25317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:19.385658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48002 (local address 127.24.183.195:43267)
0809 15:12:19.385912 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:19.385920 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:19.385974 (+ 54us) server_negotiation.cc:407] Connection header received
0809 15:12:19.386210 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:19.386215 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:19.386299 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:19.386439 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:12:19.386822 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:19.387570 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:19.388945 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:19.389360 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:19.389486 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:19.389947 (+ 461us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:12:19.390456 (+ 509us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:12:19.390615 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:19.390695 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 15:12:19.390800 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 15:12:19.399260 24838 raft_consensus.cc:1261] T 4087f96a425841b182a2206ca764a850 P 2e1c099e32914df38df95a77b5ecc220 [term 1 FOLLOWER]: Refusing update from remote peer a9683a2b3ae742c5b93beca3cfdf94bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:12:19.399291 25037 raft_consensus.cc:1261] T 4087f96a425841b182a2206ca764a850 P 436a238b0f8d42d2ab936da91e19e127 [term 1 FOLLOWER]: Refusing update from remote peer a9683a2b3ae742c5b93beca3cfdf94bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:12:19.400287 25335 consensus_queue.cc:1035] T 4087f96a425841b182a2206ca764a850 P a9683a2b3ae742c5b93beca3cfdf94bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e1c099e32914df38df95a77b5ecc220" member_type: VOTER last_known_addr { host: "127.24.183.193" port: 45119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:12:19.400604 25333 consensus_queue.cc:1035] T 4087f96a425841b182a2206ca764a850 P a9683a2b3ae742c5b93beca3cfdf94bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "436a238b0f8d42d2ab936da91e19e127" member_type: VOTER last_known_addr { host: "127.24.183.194" port: 45769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:12:19.425976 24616 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:12:19.426350 24616 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:12:19.429980 24616 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d6281c225070499e878b61c930a794fe: Sending DeleteTablet for 3 replicas of tablet 4087f96a425841b182a2206ca764a850
I20260809 15:12:19.431236 25202 tablet_service.cc:1514] Processing DeleteTablet for tablet 4087f96a425841b182a2206ca764a850 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:12:19 UTC) from {username='slave'} at 127.0.0.1:47994
I20260809 15:12:19.431473 24810 tablet_service.cc:1514] Processing DeleteTablet for tablet 4087f96a425841b182a2206ca764a850 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:12:19 UTC) from {username='slave'} at 127.0.0.1:50786
I20260809 15:12:19.431990 25008 tablet_service.cc:1514] Processing DeleteTablet for tablet 4087f96a425841b182a2206ca764a850 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:12:19 UTC) from {username='slave'} at 127.0.0.1:36816
I20260809 15:12:19.432111 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 24714
I20260809 15:12:19.442090 25358 tablet_replica.cc:330] T 4087f96a425841b182a2206ca764a850 P 436a238b0f8d42d2ab936da91e19e127: stopping tablet replica
I20260809 15:12:19.443814 25358 raft_consensus.cc:2229] T 4087f96a425841b182a2206ca764a850 P 436a238b0f8d42d2ab936da91e19e127 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:12:19.444883 25358 raft_consensus.cc:2258] T 4087f96a425841b182a2206ca764a850 P 436a238b0f8d42d2ab936da91e19e127 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:12:19.448282 25356 tablet_replica.cc:330] T 4087f96a425841b182a2206ca764a850 P 2e1c099e32914df38df95a77b5ecc220: stopping tablet replica
I20260809 15:12:19.448458 25358 ts_tablet_manager.cc:1905] T 4087f96a425841b182a2206ca764a850 P 436a238b0f8d42d2ab936da91e19e127: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:12:19.448908 25356 raft_consensus.cc:2229] T 4087f96a425841b182a2206ca764a850 P 2e1c099e32914df38df95a77b5ecc220 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:12:19.449584 25356 raft_consensus.cc:2258] T 4087f96a425841b182a2206ca764a850 P 2e1c099e32914df38df95a77b5ecc220 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:12:19.450739 25355 tablet_replica.cc:330] T 4087f96a425841b182a2206ca764a850 P a9683a2b3ae742c5b93beca3cfdf94bd: stopping tablet replica
I20260809 15:12:19.451259 25355 raft_consensus.cc:2229] T 4087f96a425841b182a2206ca764a850 P a9683a2b3ae742c5b93beca3cfdf94bd [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:12:19.452138 25355 raft_consensus.cc:2258] T 4087f96a425841b182a2206ca764a850 P a9683a2b3ae742c5b93beca3cfdf94bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:12:19.454641 25355 ts_tablet_manager.cc:1905] T 4087f96a425841b182a2206ca764a850 P a9683a2b3ae742c5b93beca3cfdf94bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:12:19.454885 25356 ts_tablet_manager.cc:1905] T 4087f96a425841b182a2206ca764a850 P 2e1c099e32914df38df95a77b5ecc220: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:12:19.456930 25319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:19.433026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50800 (local address 127.24.183.193:45119)
0809 15:12:19.433201 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:19.433207 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:19.448075 (+ 14868us) server_negotiation.cc:407] Connection header received
0809 15:12:19.448279 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:19.448286 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:19.448414 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:19.448526 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:19.449232 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:19.450569 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:19.452499 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:19.453115 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:19.453218 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:19.454771 (+ 1553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:19.454783 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:19.454801 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:19.455589 (+ 788us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:19.455653 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:19.455729 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:19.455835 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:19.456447 (+ 612us) server_negotiation.cc:299] Negotiation successful
0809 15:12:19.456571 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 15:12:19.458262 24904 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:12:19.461004 25358 ts_tablet_manager.cc:1918] T 4087f96a425841b182a2206ca764a850 P 436a238b0f8d42d2ab936da91e19e127: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:12:19.461264 25358 log.cc:1196] T 4087f96a425841b182a2206ca764a850 P 436a238b0f8d42d2ab936da91e19e127: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-1/wal/wals/4087f96a425841b182a2206ca764a850
I20260809 15:12:19.461902 25358 ts_tablet_manager.cc:1939] T 4087f96a425841b182a2206ca764a850 P 436a238b0f8d42d2ab936da91e19e127: Deleting consensus metadata
I20260809 15:12:19.462466 25355 ts_tablet_manager.cc:1918] T 4087f96a425841b182a2206ca764a850 P a9683a2b3ae742c5b93beca3cfdf94bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:12:19.462713 25355 log.cc:1196] T 4087f96a425841b182a2206ca764a850 P a9683a2b3ae742c5b93beca3cfdf94bd: Deleting WAL directory at /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288021313853-25311-0/minicluster-data/ts-2/wal/wals/4087f96a425841b182a2206ca764a850
I20260809 15:12:19.463351 25355 ts_tablet_manager.cc:1939] T 4087f96a425841b182a2206ca764a850 P a9683a2b3ae742c5b93beca3cfdf94bd: Deleting consensus metadata
I20260809 15:12:19.464144 24602 catalog_manager.cc:4841] TS 436a238b0f8d42d2ab936da91e19e127 (127.24.183.194:45769): tablet 4087f96a425841b182a2206ca764a850 (table test-table [id=9d50692bb7aa45dda3aefc451e425e94]) successfully deleted
I20260809 15:12:19.465168 24602 catalog_manager.cc:4841] TS a9683a2b3ae742c5b93beca3cfdf94bd (127.24.183.195:43267): tablet 4087f96a425841b182a2206ca764a850 (table test-table [id=9d50692bb7aa45dda3aefc451e425e94]) successfully deleted
I20260809 15:12:19.568598 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 24714
W20260809 15:12:19.597872 24601 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.183.193:45119 (error 108)
I20260809 15:12:19.598140 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 24917
I20260809 15:12:19.600426 25363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:19.599563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:45119 (local address 127.0.0.1:50806)
0809 15:12:19.600066 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:19.600212 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:45119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":296,"threads_started":1}
W20260809 15:12:19.600819 24601 catalog_manager.cc:4584] TS 2e1c099e32914df38df95a77b5ecc220 (127.24.183.193:45119): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4087f96a425841b182a2206ca764a850: Network error: Client connection negotiation failed: client connection to 127.24.183.193:45119: connect: Connection refused (error 111)
I20260809 15:12:19.605594 25321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:19.599267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36820 (local address 127.24.183.194:45769)
0809 15:12:19.599446 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:19.599452 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:19.599475 (+ 23us) server_negotiation.cc:407] Connection header received
0809 15:12:19.599547 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:19.599553 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:19.599664 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:19.599827 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:19.600436 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:19.601397 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:19.602642 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:19.603055 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:19.603167 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:19.604254 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:19.604264 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:19.604279 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:19.604864 (+ 585us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:19.604913 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:19.604933 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:19.605042 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:19.605165 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 15:12:19.605273 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 15:12:19.606209 25095 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:12:19.654474 25363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:19.653917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:45119 (local address 127.0.0.1:50818)
0809 15:12:19.654163 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:19.654277 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:45119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260809 15:12:19.655759 25363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:19.655306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:45119 (local address 127.0.0.1:50824)
0809 15:12:19.655465 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:19.655568 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:45119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 15:12:19.697202 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 24917
I20260809 15:12:19.716281 25363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:19.715827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:45119 (local address 127.0.0.1:50830)
0809 15:12:19.716030 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:19.716118 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:45119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 15:12:19.717403 25363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:19.717018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:45119 (local address 127.0.0.1:50832)
0809 15:12:19.717179 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:19.717249 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:45119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 15:12:19.727100 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 25109
I20260809 15:12:19.734807 25317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:19.727908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48014 (local address 127.24.183.195:43267)
0809 15:12:19.728105 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:19.728111 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:19.728139 (+ 28us) server_negotiation.cc:407] Connection header received
0809 15:12:19.728261 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:19.728267 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:19.728376 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:19.728502 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:19.728873 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:19.730069 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:19.731256 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:19.731949 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:19.732090 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:19.733100 (+ 1010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:19.733112 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:19.733134 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:19.734020 (+ 886us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:19.734085 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:19.734114 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:19.734242 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:19.734305 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 15:12:19.734419 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 15:12:19.735556 25287 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:12:19.798173 25363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:19.797639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:45119 (local address 127.0.0.1:50834)
0809 15:12:19.797894 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:19.798015 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:45119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260809 15:12:19.799294 25363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:19.798885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:45119 (local address 127.0.0.1:50842)
0809 15:12:19.799071 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:19.799144 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:45119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260809 15:12:19.858150 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 25109
I20260809 15:12:19.882045 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 24574
I20260809 15:12:19.883208 24671 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:12:19.994484 25363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:19.993856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.193:45119 (local address 127.0.0.1:50854)
0809 15:12:19.994140 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:19.994271 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.193:45119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147}
I20260809 15:12:19.994673 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 24574
2026-08-09T15:12:20Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3022 ms)
[----------] 6 tests from EncryptionPolicyTest (22133 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:12:20 dist-test-slave-nj21 krb5kdc[25372](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:12:20 dist-test-slave-nj21 krb5kdc[25372](info): set up 2 sockets
Aug 09 15:12:20 dist-test-slave-nj21 krb5kdc[25372](info): commencing operation
krb5kdc: starting...
W20260809 15:12:21.227164 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:12:21 dist-test-slave-nj21 krb5kdc[25372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:12:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:12:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 15:12:21.514916 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45305
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45305
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:12:21.856532 25392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:21.856802 25392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:21.856923 25392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:21.863970 25392 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:12:21.864068 25392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:21.864140 25392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:21.864219 25392 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:12:21.864279 25392 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:12:21.873894 25392 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45305
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:21.875106 25392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:21.876433 25392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:21.887799 25404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:21.887774 25407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:21.887854 25402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:21.888646 25392 server_base.cc:1034] running on GCE node
I20260809 15:12:21.889427 25392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:21.890914 25392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:21.892115 25392 hybrid_clock.cc:648] HybridClock initialized: now 1786288341892062 us; error 42 us; skew 500 ppm
I20260809 15:12:21.892454 25392 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:12:21 dist-test-slave-nj21 krb5kdc[25372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288341, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:12:21.895601 25392 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 15:12:21.897181 25392 webserver.cc:460] Webserver started at http://127.0.0.1:32983/ using document root <none> and password file <none>
I20260809 15:12:21.897908 25392 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:21.898039 25392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:21.898347 25392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:21.901471 25392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "861800bb61f34b69ba882a001294eadf"
format_stamp: "Formatted at 2026-08-09 15:12:21 on dist-test-slave-nj21"
server_key: "e08d4b1a4c6414d191eac1aba6209520"
server_key_iv: "bd9972f759d9b8d9a7dd74f4a17b883b"
server_key_version: "encryptionkey@0"
I20260809 15:12:21.902279 25392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "861800bb61f34b69ba882a001294eadf"
format_stamp: "Formatted at 2026-08-09 15:12:21 on dist-test-slave-nj21"
server_key: "e08d4b1a4c6414d191eac1aba6209520"
server_key_iv: "bd9972f759d9b8d9a7dd74f4a17b883b"
server_key_version: "encryptionkey@0"
I20260809 15:12:21.907447 25392 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 15:12:21.910894 25417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:21.912230 25392 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:12:21.912470 25392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "861800bb61f34b69ba882a001294eadf"
format_stamp: "Formatted at 2026-08-09 15:12:21 on dist-test-slave-nj21"
server_key: "e08d4b1a4c6414d191eac1aba6209520"
server_key_iv: "bd9972f759d9b8d9a7dd74f4a17b883b"
server_key_version: "encryptionkey@0"
I20260809 15:12:21.912678 25392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:21.939128 25392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:21.943387 25392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:21.943651 25392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:21.960886 25392 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45305
I20260809 15:12:21.960908 25484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45305 every 8 connection(s)
I20260809 15:12:21.962181 25392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:12:21.962951 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 25392
I20260809 15:12:21.963165 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:12:21.963912 25311 external_mini_cluster.cc:1428] Setting key caa76130664e3efbbbc0eb818c0abf0a
I20260809 15:12:21.966593 25485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 15:12:21 dist-test-slave-nj21 krb5kdc[25372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 15:12:21.978999 25485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 861800bb61f34b69ba882a001294eadf: Bootstrap starting.
I20260809 15:12:21.982923 25485 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 861800bb61f34b69ba882a001294eadf: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:21.984155 25485 log.cc:824] T 00000000000000000000000000000000 P 861800bb61f34b69ba882a001294eadf: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:21.987413 25485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 861800bb61f34b69ba882a001294eadf: No bootstrap required, opened a new log
I20260809 15:12:21.987767 25489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:21.965760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59033 (local address 127.0.0.1:45305)
0809 15:12:21.966607 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:21.966623 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:21.966678 (+ 55us) server_negotiation.cc:407] Connection header received
0809 15:12:21.969035 (+ 2357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:21.969061 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:21.969540 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:21.969978 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:21.971310 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:21.973363 (+ 2053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:21.974553 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:21.975549 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:21.975756 (+ 207us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:21.979580 (+ 3824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:21.979590 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:21.979606 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:12:21.979653 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:21.983507 (+ 3854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:21.984249 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:21.984258 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:21.984266 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:21.984464 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:21.984937 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:21.984940 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:21.984942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:21.985447 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:21.985769 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:21.986156 (+ 387us) server_negotiation.cc:299] Negotiation successful
0809 15:12:21.986465 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":325,"threads_started":1}
I20260809 15:12:21.991470 25311 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:12:21.996860 25485 raft_consensus.cc:348] T 00000000000000000000000000000000 P 861800bb61f34b69ba882a001294eadf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861800bb61f34b69ba882a001294eadf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45305 } }
I20260809 15:12:21.997264 25485 raft_consensus.cc:374] T 00000000000000000000000000000000 P 861800bb61f34b69ba882a001294eadf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:21.997459 25485 raft_consensus.cc:729] T 00000000000000000000000000000000 P 861800bb61f34b69ba882a001294eadf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 861800bb61f34b69ba882a001294eadf, State: Initialized, Role: FOLLOWER
I20260809 15:12:21.998070 25485 consensus_queue.cc:260] T 00000000000000000000000000000000 P 861800bb61f34b69ba882a001294eadf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861800bb61f34b69ba882a001294eadf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45305 } }
I20260809 15:12:21.998354 25485 raft_consensus.cc:388] T 00000000000000000000000000000000 P 861800bb61f34b69ba882a001294eadf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:12:21.998494 25485 raft_consensus.cc:482] T 00000000000000000000000000000000 P 861800bb61f34b69ba882a001294eadf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:12:21.998653 25485 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 861800bb61f34b69ba882a001294eadf [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:22.001387 25485 raft_consensus.cc:504] T 00000000000000000000000000000000 P 861800bb61f34b69ba882a001294eadf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861800bb61f34b69ba882a001294eadf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45305 } }
I20260809 15:12:22.001861 25485 leader_election.cc:302] T 00000000000000000000000000000000 P 861800bb61f34b69ba882a001294eadf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 861800bb61f34b69ba882a001294eadf; no voters:
I20260809 15:12:22.002461 25485 leader_election.cc:288] T 00000000000000000000000000000000 P 861800bb61f34b69ba882a001294eadf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:12:22.002552 25496 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 861800bb61f34b69ba882a001294eadf [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:12:22.004865 25485 sys_catalog.cc:564] T 00000000000000000000000000000000 P 861800bb61f34b69ba882a001294eadf [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:12:22.006377 25496 raft_consensus.cc:686] T 00000000000000000000000000000000 P 861800bb61f34b69ba882a001294eadf [term 1 LEADER]: Becoming Leader. State: Replica: 861800bb61f34b69ba882a001294eadf, State: Running, Role: LEADER
I20260809 15:12:22.007022 25496 consensus_queue.cc:237] T 00000000000000000000000000000000 P 861800bb61f34b69ba882a001294eadf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861800bb61f34b69ba882a001294eadf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45305 } }
I20260809 15:12:22.010535 25498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 861800bb61f34b69ba882a001294eadf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "861800bb61f34b69ba882a001294eadf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861800bb61f34b69ba882a001294eadf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45305 } } }
I20260809 15:12:22.010861 25498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 861800bb61f34b69ba882a001294eadf [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:22.010177 25502 sys_catalog.cc:455] T 00000000000000000000000000000000 P 861800bb61f34b69ba882a001294eadf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 861800bb61f34b69ba882a001294eadf. Latest consensus state: current_term: 1 leader_uuid: "861800bb61f34b69ba882a001294eadf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861800bb61f34b69ba882a001294eadf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45305 } } }
I20260809 15:12:22.011359 25502 sys_catalog.cc:458] T 00000000000000000000000000000000 P 861800bb61f34b69ba882a001294eadf [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:22.012935 25513 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:12:22.014185 25489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:21.997212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52675 (local address 127.0.0.1:45305)
0809 15:12:21.997407 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:21.997413 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:21.997784 (+ 371us) server_negotiation.cc:407] Connection header received
0809 15:12:21.997965 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:21.997971 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:21.998044 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:21.998156 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:21.999599 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:22.000777 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:22.002016 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:22.002705 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:22.002859 (+ 154us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:22.005934 (+ 3075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:22.005941 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:22.005947 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:12:22.005992 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:22.010036 (+ 4044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:22.011111 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:22.011118 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:22.011122 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:22.011366 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:22.012023 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:22.012032 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:22.012036 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:22.012481 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:22.012821 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:22.013642 (+ 821us) server_negotiation.cc:299] Negotiation successful
0809 15:12:22.013847 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 15:12:22.019078 25513 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:12:22.026686 25513 catalog_manager.cc:1347] Generated new cluster ID: 0e016a8e9fdd43c99deaf3535f55302f
I20260809 15:12:22.026786 25513 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:12:22.051057 25513 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:12:22.052385 25513 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:12:22.062325 25513 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 861800bb61f34b69ba882a001294eadf: Generated new TSK 0
I20260809 15:12:22.063155 25513 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:12:22.075275 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 25392
I20260809 15:12:22.076871 25480 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:12:22.169759 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 25392
W20260809 15:12:22.194103 25370 connection.cc:535] client connection to 127.0.0.1:45305 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45305): Connection reset by peer (error 104)
2026-08-09T15:12:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2170 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:12:22 dist-test-slave-nj21 krb5kdc[25527](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:12:22 dist-test-slave-nj21 krb5kdc[25527](info): set up 2 sockets
Aug 09 15:12:22 dist-test-slave-nj21 krb5kdc[25527](info): commencing operation
krb5kdc: starting...
W20260809 15:12:23.401376 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.008s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:12:23 dist-test-slave-nj21 krb5kdc[25527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:12:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:12:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 15:12:23.692384 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36149
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:12:24.028443 25542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:24.028703 25542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:24.028812 25542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:24.035818 25542 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:12:24.035923 25542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:24.035977 25542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:24.036038 25542 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:12:24.036080 25542 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:12:24.048368 25542 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36149
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:24.049635 25542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:24.050956 25542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:24.062311 25548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:24.062314 25553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:24.062335 25549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:24.063508 25542 server_base.cc:1034] running on GCE node
I20260809 15:12:24.064270 25542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:24.065603 25542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:24.066807 25542 hybrid_clock.cc:648] HybridClock initialized: now 1786288344066761 us; error 38 us; skew 500 ppm
I20260809 15:12:24.067134 25542 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:12:24 dist-test-slave-nj21 krb5kdc[25527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288344, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:12:24.070204 25542 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 15:12:24.071554 25542 webserver.cc:460] Webserver started at http://127.0.0.1:37251/ using document root <none> and password file <none>
I20260809 15:12:24.072212 25542 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:24.072357 25542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:24.072655 25542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:24.075595 25542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "2ba2dfa0ddd64b17ae3fcb1016288710"
format_stamp: "Formatted at 2026-08-09 15:12:24 on dist-test-slave-nj21"
server_key: "5b6e9655314c7816f2f2f4d92aecb584"
server_key_iv: "57039404b679eb9a64240ec06c8e8d26"
server_key_version: "encryptionkey@0"
I20260809 15:12:24.076402 25542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "2ba2dfa0ddd64b17ae3fcb1016288710"
format_stamp: "Formatted at 2026-08-09 15:12:24 on dist-test-slave-nj21"
server_key: "5b6e9655314c7816f2f2f4d92aecb584"
server_key_iv: "57039404b679eb9a64240ec06c8e8d26"
server_key_version: "encryptionkey@0"
I20260809 15:12:24.081367 25542 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260809 15:12:24.084678 25562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:24.085887 25542 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:12:24.086042 25542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "2ba2dfa0ddd64b17ae3fcb1016288710"
format_stamp: "Formatted at 2026-08-09 15:12:24 on dist-test-slave-nj21"
server_key: "5b6e9655314c7816f2f2f4d92aecb584"
server_key_iv: "57039404b679eb9a64240ec06c8e8d26"
server_key_version: "encryptionkey@0"
I20260809 15:12:24.086217 25542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:24.109617 25542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:24.113443 25542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:24.113711 25542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:24.131573 25542 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36149
I20260809 15:12:24.131597 25640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36149 every 8 connection(s)
I20260809 15:12:24.132894 25542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:12:24.136600 25642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:24.138247 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 25542
I20260809 15:12:24.138408 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:12:24.138898 25311 external_mini_cluster.cc:1428] Setting key 7144bc7f1b66523cd8d8def300c69fae
I20260809 15:12:24.151095 25642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ba2dfa0ddd64b17ae3fcb1016288710: Bootstrap starting.
I20260809 15:12:24.153066 25645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:24.140632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43399 (local address 127.0.0.1:36149)
0809 15:12:24.141434 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:24.141448 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:24.141499 (+ 51us) server_negotiation.cc:407] Connection header received
0809 15:12:24.143164 (+ 1665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:24.143190 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:24.143709 (+ 519us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:24.144161 (+ 452us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:24.144652 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:24.146454 (+ 1802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:24.147776 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:24.148841 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:24.149034 (+ 193us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:24.150195 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:24.150205 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:24.150222 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:24.150921 (+ 699us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:24.151017 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:24.151056 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:24.151217 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:24.151548 (+ 331us) server_negotiation.cc:299] Negotiation successful
0809 15:12:24.151755 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":328,"threads_started":1}
I20260809 15:12:24.154860 25642 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ba2dfa0ddd64b17ae3fcb1016288710: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:24.156111 25642 log.cc:824] T 00000000000000000000000000000000 P 2ba2dfa0ddd64b17ae3fcb1016288710: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:24.157153 25311 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:12:24.158952 25642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ba2dfa0ddd64b17ae3fcb1016288710: No bootstrap required, opened a new log
I20260809 15:12:24.167930 25645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:24.161460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48071 (local address 127.0.0.1:36149)
0809 15:12:24.161636 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:24.161641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:24.162132 (+ 491us) server_negotiation.cc:407] Connection header received
0809 15:12:24.162279 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:24.162284 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:24.162361 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:24.162459 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:24.162885 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:24.164078 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:24.165409 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:24.166104 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:24.166228 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:24.167180 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:24.167186 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:24.167191 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:24.167209 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:24.167283 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:24.167306 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:24.167421 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:24.167520 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 15:12:24.167569 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 15:12:24.167634 25642 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ba2dfa0ddd64b17ae3fcb1016288710 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ba2dfa0ddd64b17ae3fcb1016288710" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36149 } }
I20260809 15:12:24.168128 25642 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ba2dfa0ddd64b17ae3fcb1016288710 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:24.168231 25642 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ba2dfa0ddd64b17ae3fcb1016288710 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba2dfa0ddd64b17ae3fcb1016288710, State: Initialized, Role: FOLLOWER
I20260809 15:12:24.168849 25642 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ba2dfa0ddd64b17ae3fcb1016288710 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ba2dfa0ddd64b17ae3fcb1016288710" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36149 } }
I20260809 15:12:24.169138 25642 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ba2dfa0ddd64b17ae3fcb1016288710 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:12:24.169269 25642 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ba2dfa0ddd64b17ae3fcb1016288710 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:12:24.169428 25642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ba2dfa0ddd64b17ae3fcb1016288710 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:24.171532 25642 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ba2dfa0ddd64b17ae3fcb1016288710 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ba2dfa0ddd64b17ae3fcb1016288710" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36149 } }
I20260809 15:12:24.172012 25642 leader_election.cc:302] T 00000000000000000000000000000000 P 2ba2dfa0ddd64b17ae3fcb1016288710 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ba2dfa0ddd64b17ae3fcb1016288710; no voters:
I20260809 15:12:24.172578 25642 leader_election.cc:288] T 00000000000000000000000000000000 P 2ba2dfa0ddd64b17ae3fcb1016288710 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:12:24.172690 25655 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ba2dfa0ddd64b17ae3fcb1016288710 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:12:24.173326 25655 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ba2dfa0ddd64b17ae3fcb1016288710 [term 1 LEADER]: Becoming Leader. State: Replica: 2ba2dfa0ddd64b17ae3fcb1016288710, State: Running, Role: LEADER
I20260809 15:12:24.173918 25655 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ba2dfa0ddd64b17ae3fcb1016288710 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ba2dfa0ddd64b17ae3fcb1016288710" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36149 } }
I20260809 15:12:24.174763 25642 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ba2dfa0ddd64b17ae3fcb1016288710 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:12:24.176868 25658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ba2dfa0ddd64b17ae3fcb1016288710 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ba2dfa0ddd64b17ae3fcb1016288710. Latest consensus state: current_term: 1 leader_uuid: "2ba2dfa0ddd64b17ae3fcb1016288710" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ba2dfa0ddd64b17ae3fcb1016288710" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36149 } } }
I20260809 15:12:24.177407 25658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ba2dfa0ddd64b17ae3fcb1016288710 [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:24.176890 25656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ba2dfa0ddd64b17ae3fcb1016288710 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ba2dfa0ddd64b17ae3fcb1016288710" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ba2dfa0ddd64b17ae3fcb1016288710" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36149 } } }
I20260809 15:12:24.177693 25656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ba2dfa0ddd64b17ae3fcb1016288710 [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:24.178719 25666 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:12:24.184523 25666 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:12:24.193516 25666 catalog_manager.cc:1347] Generated new cluster ID: ce547e553b3a4676927c19b948529e6b
I20260809 15:12:24.193611 25666 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:12:24.202731 25666 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:12:24.204241 25666 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:12:24.215616 25666 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ba2dfa0ddd64b17ae3fcb1016288710: Generated new TSK 0
I20260809 15:12:24.216370 25666 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:12:24.229312 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 25542
W20260809 15:12:24.230680 25634 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:43399
W20260809 15:12:24.231433 25311 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 25542: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:12:24.231600 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 25542
W20260809 15:12:24.255200 25524 connection.cc:535] client connection to 127.0.0.1:36149 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36149): Connection reset by peer (error 104)
2026-08-09T15:12:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2060 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:12:24 dist-test-slave-nj21 krb5kdc[25683](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:12:24 dist-test-slave-nj21 krb5kdc[25683](info): set up 2 sockets
Aug 09 15:12:24 dist-test-slave-nj21 krb5kdc[25683](info): commencing operation
krb5kdc: starting...
W20260809 15:12:25.445634 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.171s user 0.001s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:12:25 dist-test-slave-nj21 krb5kdc[25683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:12:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:12:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 15:12:25.727710 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35853
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35853
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:12:26.069304 25704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:26.069609 25704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:26.069763 25704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:26.077068 25704 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:12:26.077189 25704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:26.077267 25704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:26.077350 25704 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:12:26.077414 25704 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:12:26.087217 25704 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35853
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:26.088580 25704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:26.089956 25704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:26.101115 25709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:26.101214 25710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:26.101316 25713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:26.102164 25704 server_base.cc:1034] running on GCE node
I20260809 15:12:26.102932 25704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:26.104414 25704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:26.105626 25704 hybrid_clock.cc:648] HybridClock initialized: now 1786288346105579 us; error 36 us; skew 500 ppm
I20260809 15:12:26.105973 25704 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:12:26 dist-test-slave-nj21 krb5kdc[25683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288346, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:12:26.109133 25704 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 15:12:26.110572 25704 webserver.cc:460] Webserver started at http://127.0.0.1:41169/ using document root <none> and password file <none>
I20260809 15:12:26.111193 25704 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:26.111297 25704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:26.111555 25704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:26.114512 25704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "cf690dec7bd74dc584ff5256346ac5c2"
format_stamp: "Formatted at 2026-08-09 15:12:26 on dist-test-slave-nj21"
server_key: "3d92885b1d66c7db590ac59335845a15"
server_key_iv: "d2792d5810623237a07288d187fcb805"
server_key_version: "encryptionkey@0"
I20260809 15:12:26.115270 25704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "cf690dec7bd74dc584ff5256346ac5c2"
format_stamp: "Formatted at 2026-08-09 15:12:26 on dist-test-slave-nj21"
server_key: "3d92885b1d66c7db590ac59335845a15"
server_key_iv: "d2792d5810623237a07288d187fcb805"
server_key_version: "encryptionkey@0"
I20260809 15:12:26.120180 25704 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 15:12:26.123436 25722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:26.124699 25704 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:12:26.124852 25704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "cf690dec7bd74dc584ff5256346ac5c2"
format_stamp: "Formatted at 2026-08-09 15:12:26 on dist-test-slave-nj21"
server_key: "3d92885b1d66c7db590ac59335845a15"
server_key_iv: "d2792d5810623237a07288d187fcb805"
server_key_version: "encryptionkey@0"
I20260809 15:12:26.125025 25704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:26.130682 25704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:26.134795 25704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:26.135073 25704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:26.152839 25704 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35853
I20260809 15:12:26.152911 25796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35853 every 8 connection(s)
I20260809 15:12:26.154230 25704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:12:26.157945 25797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:26.164440 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 25704
I20260809 15:12:26.164644 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:12:26.165243 25311 external_mini_cluster.cc:1428] Setting key 17b8a271374cedf17320efb91fae703f
I20260809 15:12:26.168797 25797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf690dec7bd74dc584ff5256346ac5c2: Bootstrap starting.
I20260809 15:12:26.172278 25797 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf690dec7bd74dc584ff5256346ac5c2: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:26.173041 25806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:26.167122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43205 (local address 127.0.0.1:35853)
0809 15:12:26.167961 (+ 839us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:26.167972 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:26.168015 (+ 43us) server_negotiation.cc:407] Connection header received
0809 15:12:26.169761 (+ 1746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:26.169784 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:26.170093 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:26.170428 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:26.171416 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:26.171423 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:26.171435 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:26.171449 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:26.171513 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:26.171533 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:26.171694 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:26.171880 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 15:12:26.172061 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":356,"threads_started":1}
I20260809 15:12:26.173363 25797 log.cc:824] T 00000000000000000000000000000000 P cf690dec7bd74dc584ff5256346ac5c2: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:26.176561 25797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf690dec7bd74dc584ff5256346ac5c2: No bootstrap required, opened a new log
I20260809 15:12:26.176843 25311 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:12:26.183362 25806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:26.181144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43953 (local address 127.0.0.1:35853)
0809 15:12:26.181319 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:26.181324 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:26.181603 (+ 279us) server_negotiation.cc:407] Connection header received
0809 15:12:26.181782 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:26.181786 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:26.181836 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:26.181913 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:26.182763 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:26.182767 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:26.182770 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:26.182785 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:26.182823 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:26.182837 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:26.182932 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:26.183040 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 15:12:26.183085 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 15:12:26.184975 25797 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf690dec7bd74dc584ff5256346ac5c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf690dec7bd74dc584ff5256346ac5c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35853 } }
I20260809 15:12:26.185312 25797 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf690dec7bd74dc584ff5256346ac5c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:26.185420 25797 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf690dec7bd74dc584ff5256346ac5c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf690dec7bd74dc584ff5256346ac5c2, State: Initialized, Role: FOLLOWER
I20260809 15:12:26.185945 25797 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf690dec7bd74dc584ff5256346ac5c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf690dec7bd74dc584ff5256346ac5c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35853 } }
I20260809 15:12:26.186172 25797 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf690dec7bd74dc584ff5256346ac5c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:12:26.186300 25797 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf690dec7bd74dc584ff5256346ac5c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:12:26.186462 25797 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf690dec7bd74dc584ff5256346ac5c2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:26.188687 25797 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf690dec7bd74dc584ff5256346ac5c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf690dec7bd74dc584ff5256346ac5c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35853 } }
I20260809 15:12:26.189169 25797 leader_election.cc:302] T 00000000000000000000000000000000 P cf690dec7bd74dc584ff5256346ac5c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf690dec7bd74dc584ff5256346ac5c2; no voters:
I20260809 15:12:26.189842 25797 leader_election.cc:288] T 00000000000000000000000000000000 P cf690dec7bd74dc584ff5256346ac5c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:12:26.189946 25815 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf690dec7bd74dc584ff5256346ac5c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:12:26.190577 25815 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf690dec7bd74dc584ff5256346ac5c2 [term 1 LEADER]: Becoming Leader. State: Replica: cf690dec7bd74dc584ff5256346ac5c2, State: Running, Role: LEADER
I20260809 15:12:26.191154 25815 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf690dec7bd74dc584ff5256346ac5c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf690dec7bd74dc584ff5256346ac5c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35853 } }
I20260809 15:12:26.191944 25797 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf690dec7bd74dc584ff5256346ac5c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:12:26.193977 25819 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf690dec7bd74dc584ff5256346ac5c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf690dec7bd74dc584ff5256346ac5c2. Latest consensus state: current_term: 1 leader_uuid: "cf690dec7bd74dc584ff5256346ac5c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf690dec7bd74dc584ff5256346ac5c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35853 } } }
I20260809 15:12:26.194495 25819 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf690dec7bd74dc584ff5256346ac5c2 [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:26.194056 25816 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf690dec7bd74dc584ff5256346ac5c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf690dec7bd74dc584ff5256346ac5c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf690dec7bd74dc584ff5256346ac5c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35853 } } }
I20260809 15:12:26.195276 25816 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf690dec7bd74dc584ff5256346ac5c2 [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:26.204535 25829 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:12:26.208729 25829 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:12:26.216892 25829 catalog_manager.cc:1347] Generated new cluster ID: 3997dc7ea20d4393b8c95fb65d3b57a1
I20260809 15:12:26.216993 25829 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:12:26.233194 25829 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:12:26.233594 25829 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:12:26.239456 25829 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf690dec7bd74dc584ff5256346ac5c2: Generated new TSK 0
I20260809 15:12:26.240324 25829 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:12:26.245101 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 25704
W20260809 15:12:26.246847 25791 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:43205
W20260809 15:12:26.247539 25311 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 25704: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:12:26.247717 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 25704
2026-08-09T15:12:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2016 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:12:26 dist-test-slave-nj21 krb5kdc[25849](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:12:26 dist-test-slave-nj21 krb5kdc[25849](info): set up 2 sockets
Aug 09 15:12:26 dist-test-slave-nj21 krb5kdc[25849](info): commencing operation
krb5kdc: starting...
W20260809 15:12:27.466369 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.005s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:12:27 dist-test-slave-nj21 krb5kdc[25849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:12:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:12:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 15:12:27.751087 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42119
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42119
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:12:28.092442 25870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:28.092746 25870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:28.092898 25870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:28.100075 25870 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:12:28.100198 25870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:28.100282 25870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:28.100371 25870 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:12:28.100435 25870 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:12:28.110277 25870 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42119
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:28.111594 25870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:28.112923 25870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:28.124647 25875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:28.124744 25876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:28.124784 25878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:28.125847 25870 server_base.cc:1034] running on GCE node
I20260809 15:12:28.126578 25870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:28.127941 25870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:28.129156 25870 hybrid_clock.cc:648] HybridClock initialized: now 1786288348129102 us; error 41 us; skew 500 ppm
I20260809 15:12:28.129490 25870 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:12:28 dist-test-slave-nj21 krb5kdc[25849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288348, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:12:28.136117 25870 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 15:12:28.137647 25870 webserver.cc:460] Webserver started at http://127.0.0.1:43447/ using document root <none> and password file <none>
I20260809 15:12:28.138322 25870 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:28.138437 25870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:28.138747 25870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:28.141894 25870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "4cb365806b464073bf1150c5ce95e13a"
format_stamp: "Formatted at 2026-08-09 15:12:28 on dist-test-slave-nj21"
server_key: "de652059fde08f81bc807927ee7b5478"
server_key_iv: "8d897b206d01baabcf72363df248a254"
server_key_version: "encryptionkey@0"
I20260809 15:12:28.142699 25870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "4cb365806b464073bf1150c5ce95e13a"
format_stamp: "Formatted at 2026-08-09 15:12:28 on dist-test-slave-nj21"
server_key: "de652059fde08f81bc807927ee7b5478"
server_key_iv: "8d897b206d01baabcf72363df248a254"
server_key_version: "encryptionkey@0"
I20260809 15:12:28.147987 25870 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 15:12:28.151532 25890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:28.152930 25870 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:12:28.153095 25870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "4cb365806b464073bf1150c5ce95e13a"
format_stamp: "Formatted at 2026-08-09 15:12:28 on dist-test-slave-nj21"
server_key: "de652059fde08f81bc807927ee7b5478"
server_key_iv: "8d897b206d01baabcf72363df248a254"
server_key_version: "encryptionkey@0"
I20260809 15:12:28.153270 25870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:28.178496 25870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:28.182751 25870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:28.183035 25870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:28.201197 25870 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42119
I20260809 15:12:28.201246 25964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42119 every 8 connection(s)
I20260809 15:12:28.202768 25870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:12:28.207026 25965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:28.211004 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 25870
I20260809 15:12:28.211205 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:12:28.211809 25311 external_mini_cluster.cc:1428] Setting key f44f0a73d7caa5ab96aa530dc4517e52
I20260809 15:12:28.220201 25965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cb365806b464073bf1150c5ce95e13a: Bootstrap starting.
I20260809 15:12:28.224071 25965 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4cb365806b464073bf1150c5ce95e13a: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:28.225389 25965 log.cc:824] T 00000000000000000000000000000000 P 4cb365806b464073bf1150c5ce95e13a: Log is configured to *not* fsync() on all Append() calls
Aug 09 15:12:28 dist-test-slave-nj21 krb5kdc[25849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 15:12:28.228322 25965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cb365806b464073bf1150c5ce95e13a: No bootstrap required, opened a new log
I20260809 15:12:28.234741 25968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:28.213670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48961 (local address 127.0.0.1:42119)
0809 15:12:28.214443 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:28.214455 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:28.214499 (+ 44us) server_negotiation.cc:407] Connection header received
0809 15:12:28.216198 (+ 1699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:28.216221 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:28.216632 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:28.216946 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:28.218783 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:28.220169 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:28.221922 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:28.222592 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:28.222718 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:28.227614 (+ 4896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:28.227623 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:28.227636 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:12:28.227703 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:28.230762 (+ 3059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:28.231422 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:28.231428 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:28.231434 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:28.231607 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:28.232015 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:28.232019 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:28.232021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:28.232563 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:28.232862 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:28.233308 (+ 446us) server_negotiation.cc:299] Negotiation successful
0809 15:12:28.233730 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":346,"threads_started":1}
I20260809 15:12:28.236274 25965 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4cb365806b464073bf1150c5ce95e13a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cb365806b464073bf1150c5ce95e13a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42119 } }
I20260809 15:12:28.236631 25965 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4cb365806b464073bf1150c5ce95e13a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:28.236737 25965 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4cb365806b464073bf1150c5ce95e13a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cb365806b464073bf1150c5ce95e13a, State: Initialized, Role: FOLLOWER
I20260809 15:12:28.237306 25965 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4cb365806b464073bf1150c5ce95e13a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cb365806b464073bf1150c5ce95e13a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42119 } }
I20260809 15:12:28.237567 25965 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4cb365806b464073bf1150c5ce95e13a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:12:28.237695 25965 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4cb365806b464073bf1150c5ce95e13a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:12:28.237869 25965 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4cb365806b464073bf1150c5ce95e13a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:28.238252 25311 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:12:28.240926 25965 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4cb365806b464073bf1150c5ce95e13a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cb365806b464073bf1150c5ce95e13a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42119 } }
I20260809 15:12:28.241492 25965 leader_election.cc:302] T 00000000000000000000000000000000 P 4cb365806b464073bf1150c5ce95e13a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4cb365806b464073bf1150c5ce95e13a; no voters:
I20260809 15:12:28.242190 25965 leader_election.cc:288] T 00000000000000000000000000000000 P 4cb365806b464073bf1150c5ce95e13a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:12:28.242295 25975 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4cb365806b464073bf1150c5ce95e13a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:12:28.243477 25975 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4cb365806b464073bf1150c5ce95e13a [term 1 LEADER]: Becoming Leader. State: Replica: 4cb365806b464073bf1150c5ce95e13a, State: Running, Role: LEADER
I20260809 15:12:28.244532 25965 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4cb365806b464073bf1150c5ce95e13a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:12:28.244589 25975 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4cb365806b464073bf1150c5ce95e13a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cb365806b464073bf1150c5ce95e13a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42119 } }
I20260809 15:12:28.247352 25980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cb365806b464073bf1150c5ce95e13a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4cb365806b464073bf1150c5ce95e13a. Latest consensus state: current_term: 1 leader_uuid: "4cb365806b464073bf1150c5ce95e13a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cb365806b464073bf1150c5ce95e13a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42119 } } }
I20260809 15:12:28.247738 25980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cb365806b464073bf1150c5ce95e13a [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:28.248291 25979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cb365806b464073bf1150c5ce95e13a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4cb365806b464073bf1150c5ce95e13a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cb365806b464073bf1150c5ce95e13a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42119 } } }
I20260809 15:12:28.248585 25979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cb365806b464073bf1150c5ce95e13a [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:28.250748 25989 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:12:28.257943 25989 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:12:28.260707 25968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:28.243953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60855 (local address 127.0.0.1:42119)
0809 15:12:28.244118 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:28.244123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:28.244431 (+ 308us) server_negotiation.cc:407] Connection header received
0809 15:12:28.244603 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:28.244607 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:28.244693 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:28.244798 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:28.246713 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:28.247853 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:28.249680 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:28.250410 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:28.250543 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:28.253371 (+ 2828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:28.253378 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:28.253382 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:12:28.253411 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:28.257037 (+ 3626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:28.258134 (+ 1097us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:28.258139 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:28.258143 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:28.258292 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:28.258795 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:28.258801 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:28.258804 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:28.259224 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:28.259469 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:28.260087 (+ 618us) server_negotiation.cc:299] Negotiation successful
0809 15:12:28.260320 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 15:12:28.268998 25989 catalog_manager.cc:1347] Generated new cluster ID: e3862b9709d84db191033e3b336c9a94
I20260809 15:12:28.269107 25989 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:12:28.287712 25989 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:12:28.289072 25989 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:12:28.295118 25989 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4cb365806b464073bf1150c5ce95e13a: Generated new TSK 0
I20260809 15:12:28.296018 25989 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:12:28.308885 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 25870
I20260809 15:12:28.310092 25960 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:12:28.403162 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 25870
2026-08-09T15:12:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2153 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:12:28 dist-test-slave-nj21 krb5kdc[26016](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:12:28 dist-test-slave-nj21 krb5kdc[26016](info): set up 2 sockets
Aug 09 15:12:28 dist-test-slave-nj21 krb5kdc[26016](info): commencing operation
krb5kdc: starting...
W20260809 15:12:29.632561 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.008s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:12:29 dist-test-slave-nj21 krb5kdc[26016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:12:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:12:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 15:12:29.918748 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40037
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40037
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:12:30.250595 26039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:30.250859 26039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:30.250988 26039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:30.258047 26039 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:12:30.258152 26039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:30.258227 26039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:30.258304 26039 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:12:30.258368 26039 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:12:30.267956 26039 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40037
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:30.269174 26039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:30.270450 26039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:30.281788 26048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:30.281692 26055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:30.281728 26052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:30.282755 26039 server_base.cc:1034] running on GCE node
I20260809 15:12:30.283488 26039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:30.284910 26039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:30.286108 26039 hybrid_clock.cc:648] HybridClock initialized: now 1786288350286064 us; error 36 us; skew 500 ppm
I20260809 15:12:30.286432 26039 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:12:30 dist-test-slave-nj21 krb5kdc[26016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288350, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:12:30.289424 26039 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 15:12:30.290874 26039 webserver.cc:460] Webserver started at http://127.0.0.1:42005/ using document root <none> and password file <none>
I20260809 15:12:30.291513 26039 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:30.291631 26039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:30.291944 26039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:30.294840 26039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "6b0ef7595992498c8bab0fc240e9420a"
format_stamp: "Formatted at 2026-08-09 15:12:30 on dist-test-slave-nj21"
server_key: "28a7ba4f8e5c4a646105b6a028bd923e"
server_key_iv: "143ad3330d9f5c8bca0327c95ef786f7"
server_key_version: "encryptionkey@0"
I20260809 15:12:30.295576 26039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "6b0ef7595992498c8bab0fc240e9420a"
format_stamp: "Formatted at 2026-08-09 15:12:30 on dist-test-slave-nj21"
server_key: "28a7ba4f8e5c4a646105b6a028bd923e"
server_key_iv: "143ad3330d9f5c8bca0327c95ef786f7"
server_key_version: "encryptionkey@0"
I20260809 15:12:30.300529 26039 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 15:12:30.303790 26065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:30.305037 26039 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 15:12:30.305207 26039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "6b0ef7595992498c8bab0fc240e9420a"
format_stamp: "Formatted at 2026-08-09 15:12:30 on dist-test-slave-nj21"
server_key: "28a7ba4f8e5c4a646105b6a028bd923e"
server_key_iv: "143ad3330d9f5c8bca0327c95ef786f7"
server_key_version: "encryptionkey@0"
I20260809 15:12:30.305382 26039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:30.323181 26039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:30.326877 26039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:30.327132 26039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:30.344753 26039 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40037
I20260809 15:12:30.344794 26142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40037 every 8 connection(s)
I20260809 15:12:30.346079 26039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:12:30.349749 26144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:30.352859 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 26039
I20260809 15:12:30.353055 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:12:30.353636 25311 external_mini_cluster.cc:1428] Setting key 028d9065a476604e4b2f9c8a0297b814
I20260809 15:12:30.361646 26144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b0ef7595992498c8bab0fc240e9420a: Bootstrap starting.
I20260809 15:12:30.365276 26144 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b0ef7595992498c8bab0fc240e9420a: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:30.366422 26144 log.cc:824] T 00000000000000000000000000000000 P 6b0ef7595992498c8bab0fc240e9420a: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:30.366508 26149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:30.355379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40007 (local address 127.0.0.1:40037)
0809 15:12:30.356157 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:30.356168 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:30.356211 (+ 43us) server_negotiation.cc:407] Connection header received
0809 15:12:30.357899 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:30.357922 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:30.358308 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:30.358640 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:30.359236 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:30.360590 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:30.362398 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:30.363043 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:30.363179 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:30.364373 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:30.364380 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:30.364393 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:30.364893 (+ 500us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:30.364965 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:30.364996 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:30.365078 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:30.365310 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 15:12:30.365535 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":374,"threads_started":1}
I20260809 15:12:30.369313 26144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b0ef7595992498c8bab0fc240e9420a: No bootstrap required, opened a new log
I20260809 15:12:30.370167 25311 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:12:30.377036 26144 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b0ef7595992498c8bab0fc240e9420a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b0ef7595992498c8bab0fc240e9420a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40037 } }
I20260809 15:12:30.377499 26144 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b0ef7595992498c8bab0fc240e9420a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:30.377619 26144 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b0ef7595992498c8bab0fc240e9420a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b0ef7595992498c8bab0fc240e9420a, State: Initialized, Role: FOLLOWER
I20260809 15:12:30.378229 26144 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b0ef7595992498c8bab0fc240e9420a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b0ef7595992498c8bab0fc240e9420a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40037 } }
I20260809 15:12:30.378463 26144 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b0ef7595992498c8bab0fc240e9420a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:12:30.378597 26144 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b0ef7595992498c8bab0fc240e9420a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:12:30.378744 26144 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b0ef7595992498c8bab0fc240e9420a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:30.381522 26144 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b0ef7595992498c8bab0fc240e9420a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b0ef7595992498c8bab0fc240e9420a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40037 } }
I20260809 15:12:30.381906 26149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:30.374379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55053 (local address 127.0.0.1:40037)
0809 15:12:30.374573 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:30.374580 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:30.376188 (+ 1608us) server_negotiation.cc:407] Connection header received
0809 15:12:30.376306 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:30.376336 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:30.376417 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:30.376520 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:30.376975 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:30.378005 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:30.379438 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:30.380153 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:30.380286 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:30.381228 (+ 942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:30.381234 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:30.381239 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:30.381261 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:30.381307 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:30.381338 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:30.381432 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:30.381509 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 15:12:30.381605 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 15:12:30.382205 26144 leader_election.cc:302] T 00000000000000000000000000000000 P 6b0ef7595992498c8bab0fc240e9420a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b0ef7595992498c8bab0fc240e9420a; no voters:
I20260809 15:12:30.382861 26144 leader_election.cc:288] T 00000000000000000000000000000000 P 6b0ef7595992498c8bab0fc240e9420a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:12:30.382966 26162 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b0ef7595992498c8bab0fc240e9420a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:12:30.383913 26162 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b0ef7595992498c8bab0fc240e9420a [term 1 LEADER]: Becoming Leader. State: Replica: 6b0ef7595992498c8bab0fc240e9420a, State: Running, Role: LEADER
I20260809 15:12:30.384498 26162 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b0ef7595992498c8bab0fc240e9420a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b0ef7595992498c8bab0fc240e9420a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40037 } }
I20260809 15:12:30.385342 26144 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b0ef7595992498c8bab0fc240e9420a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:12:30.387282 26165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b0ef7595992498c8bab0fc240e9420a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b0ef7595992498c8bab0fc240e9420a. Latest consensus state: current_term: 1 leader_uuid: "6b0ef7595992498c8bab0fc240e9420a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b0ef7595992498c8bab0fc240e9420a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40037 } } }
I20260809 15:12:30.387738 26165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b0ef7595992498c8bab0fc240e9420a [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:30.388245 26164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b0ef7595992498c8bab0fc240e9420a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b0ef7595992498c8bab0fc240e9420a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b0ef7595992498c8bab0fc240e9420a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40037 } } }
I20260809 15:12:30.388568 26164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b0ef7595992498c8bab0fc240e9420a [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:30.389995 26177 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:12:30.394735 26177 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:12:30.402927 26177 catalog_manager.cc:1347] Generated new cluster ID: 87295b06d9e54ce69bbb62a1414faedb
I20260809 15:12:30.403043 26177 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:12:30.413057 26177 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:12:30.414268 26177 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:12:30.436547 26177 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b0ef7595992498c8bab0fc240e9420a: Generated new TSK 0
I20260809 15:12:30.437362 26177 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:12:30.442629 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 26039
W20260809 15:12:30.443828 26135 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:40007
W20260809 15:12:30.444448 25311 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 26039: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:12:30.444569 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 26039
2026-08-09T15:12:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2043 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:12:30 dist-test-slave-nj21 krb5kdc[26200](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:12:30 dist-test-slave-nj21 krb5kdc[26200](info): set up 2 sockets
Aug 09 15:12:30 dist-test-slave-nj21 krb5kdc[26200](info): commencing operation
krb5kdc: starting...
W20260809 15:12:31.675226 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.001s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:12:31 dist-test-slave-nj21 krb5kdc[26200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:12:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:12:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 15:12:31.982925 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38283
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38283
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:12:32.321552 26220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:32.321851 26220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:32.321990 26220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:32.328943 26220 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:12:32.329056 26220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:32.329130 26220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:32.329213 26220 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:12:32.329272 26220 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:12:32.338647 26220 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38283
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:32.339879 26220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:32.341140 26220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:32.352315 26227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:32.352388 26228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:32.352404 26231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:32.353474 26220 server_base.cc:1034] running on GCE node
I20260809 15:12:32.354219 26220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:32.355542 26220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:32.356755 26220 hybrid_clock.cc:648] HybridClock initialized: now 1786288352356697 us; error 44 us; skew 500 ppm
I20260809 15:12:32.357100 26220 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:12:32 dist-test-slave-nj21 krb5kdc[26200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288352, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:12:32.360219 26220 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 15:12:32.361560 26220 webserver.cc:460] Webserver started at http://127.0.0.1:38393/ using document root <none> and password file <none>
I20260809 15:12:32.362195 26220 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:32.362316 26220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:32.362608 26220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:32.365556 26220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "974929470c024e879764dea6bb7b1b0d"
format_stamp: "Formatted at 2026-08-09 15:12:32 on dist-test-slave-nj21"
server_key: "0362d49ca4c19b7c6658572f6626ebe7"
server_key_iv: "d18b8058eafaacf55069efecd9d47eff"
server_key_version: "encryptionkey@0"
I20260809 15:12:32.366298 26220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "974929470c024e879764dea6bb7b1b0d"
format_stamp: "Formatted at 2026-08-09 15:12:32 on dist-test-slave-nj21"
server_key: "0362d49ca4c19b7c6658572f6626ebe7"
server_key_iv: "d18b8058eafaacf55069efecd9d47eff"
server_key_version: "encryptionkey@0"
I20260809 15:12:32.371184 26220 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 15:12:32.374497 26241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:32.375793 26220 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:12:32.375972 26220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "974929470c024e879764dea6bb7b1b0d"
format_stamp: "Formatted at 2026-08-09 15:12:32 on dist-test-slave-nj21"
server_key: "0362d49ca4c19b7c6658572f6626ebe7"
server_key_iv: "d18b8058eafaacf55069efecd9d47eff"
server_key_version: "encryptionkey@0"
I20260809 15:12:32.376147 26220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:32.381464 26220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:32.385129 26220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:32.385396 26220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:32.402757 26220 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38283
I20260809 15:12:32.402796 26313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38283 every 8 connection(s)
I20260809 15:12:32.404088 26220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:12:32.407647 26314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:32.409916 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 26220
I20260809 15:12:32.410102 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:12:32.410619 25311 external_mini_cluster.cc:1428] Setting key 2948feb68eebb1564c727d054c0cc1cd
I20260809 15:12:32.418385 26320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:32.412400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57873 (local address 127.0.0.1:38283)
0809 15:12:32.413196 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:32.413210 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:32.413254 (+ 44us) server_negotiation.cc:407] Connection header received
0809 15:12:32.414917 (+ 1663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:32.414940 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:32.415235 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:32.415539 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:32.416625 (+ 1086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:32.416634 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:32.416656 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:32.416676 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:32.416762 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:32.416795 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:32.417032 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:32.417184 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 15:12:32.417402 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":377,"threads_started":1}
I20260809 15:12:32.419687 26314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 974929470c024e879764dea6bb7b1b0d: Bootstrap starting.
I20260809 15:12:32.421752 25311 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:12:32.422847 26314 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 974929470c024e879764dea6bb7b1b0d: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:32.423840 26314 log.cc:824] T 00000000000000000000000000000000 P 974929470c024e879764dea6bb7b1b0d: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:32.426523 26314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 974929470c024e879764dea6bb7b1b0d: No bootstrap required, opened a new log
I20260809 15:12:32.428232 26320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:32.425921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41601 (local address 127.0.0.1:38283)
0809 15:12:32.426112 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:32.426118 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:32.426540 (+ 422us) server_negotiation.cc:407] Connection header received
0809 15:12:32.426676 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:32.426680 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:32.426737 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:32.426834 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:32.427604 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:32.427609 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:32.427613 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:32.427631 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:32.427696 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:32.427717 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:32.427825 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:32.427893 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 15:12:32.427948 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 15:12:32.434048 26314 raft_consensus.cc:348] T 00000000000000000000000000000000 P 974929470c024e879764dea6bb7b1b0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "974929470c024e879764dea6bb7b1b0d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38283 } }
I20260809 15:12:32.434382 26314 raft_consensus.cc:374] T 00000000000000000000000000000000 P 974929470c024e879764dea6bb7b1b0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:32.434456 26314 raft_consensus.cc:729] T 00000000000000000000000000000000 P 974929470c024e879764dea6bb7b1b0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 974929470c024e879764dea6bb7b1b0d, State: Initialized, Role: FOLLOWER
I20260809 15:12:32.434911 26314 consensus_queue.cc:260] T 00000000000000000000000000000000 P 974929470c024e879764dea6bb7b1b0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "974929470c024e879764dea6bb7b1b0d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38283 } }
I20260809 15:12:32.435108 26314 raft_consensus.cc:388] T 00000000000000000000000000000000 P 974929470c024e879764dea6bb7b1b0d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:12:32.435204 26314 raft_consensus.cc:482] T 00000000000000000000000000000000 P 974929470c024e879764dea6bb7b1b0d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:12:32.435322 26314 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 974929470c024e879764dea6bb7b1b0d [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:32.437449 26314 raft_consensus.cc:504] T 00000000000000000000000000000000 P 974929470c024e879764dea6bb7b1b0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "974929470c024e879764dea6bb7b1b0d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38283 } }
I20260809 15:12:32.437903 26314 leader_election.cc:302] T 00000000000000000000000000000000 P 974929470c024e879764dea6bb7b1b0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 974929470c024e879764dea6bb7b1b0d; no voters:
I20260809 15:12:32.438445 26314 leader_election.cc:288] T 00000000000000000000000000000000 P 974929470c024e879764dea6bb7b1b0d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:12:32.438568 26332 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 974929470c024e879764dea6bb7b1b0d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:12:32.439209 26332 raft_consensus.cc:686] T 00000000000000000000000000000000 P 974929470c024e879764dea6bb7b1b0d [term 1 LEADER]: Becoming Leader. State: Replica: 974929470c024e879764dea6bb7b1b0d, State: Running, Role: LEADER
I20260809 15:12:32.439810 26332 consensus_queue.cc:237] T 00000000000000000000000000000000 P 974929470c024e879764dea6bb7b1b0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "974929470c024e879764dea6bb7b1b0d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38283 } }
I20260809 15:12:32.440382 26314 sys_catalog.cc:564] T 00000000000000000000000000000000 P 974929470c024e879764dea6bb7b1b0d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:12:32.442196 26333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 974929470c024e879764dea6bb7b1b0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "974929470c024e879764dea6bb7b1b0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "974929470c024e879764dea6bb7b1b0d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38283 } } }
I20260809 15:12:32.442528 26333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 974929470c024e879764dea6bb7b1b0d [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:32.443866 26334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 974929470c024e879764dea6bb7b1b0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 974929470c024e879764dea6bb7b1b0d. Latest consensus state: current_term: 1 leader_uuid: "974929470c024e879764dea6bb7b1b0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "974929470c024e879764dea6bb7b1b0d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38283 } } }
I20260809 15:12:32.444154 26334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 974929470c024e879764dea6bb7b1b0d [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:32.444199 26338 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:12:32.449406 26338 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:12:32.457717 26338 catalog_manager.cc:1347] Generated new cluster ID: c978da2c59354b8a8f0e734d1128b28c
I20260809 15:12:32.457815 26338 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:12:32.468808 26338 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:12:32.469197 26338 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:12:32.476809 26338 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 974929470c024e879764dea6bb7b1b0d: Generated new TSK 0
I20260809 15:12:32.477483 26338 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:12:32.489358 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 26220
W20260809 15:12:32.491118 26308 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57873
W20260809 15:12:32.491786 25311 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 26220: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:12:32.491935 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 26220
2026-08-09T15:12:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2053 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:12:32 dist-test-slave-nj21 krb5kdc[26362](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:12:32 dist-test-slave-nj21 krb5kdc[26362](info): set up 2 sockets
Aug 09 15:12:32 dist-test-slave-nj21 krb5kdc[26362](info): commencing operation
krb5kdc: starting...
W20260809 15:12:33.732981 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:12:33 dist-test-slave-nj21 krb5kdc[26362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:12:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:12:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 15:12:34.030514 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35275
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35275
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:12:34.372874 26387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:34.373145 26387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:34.373291 26387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:34.380630 26387 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:12:34.380751 26387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:34.380827 26387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:34.380911 26387 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:12:34.380968 26387 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:12:34.391379 26387 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:34.393745 26387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:34.395303 26387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:34.407647 26392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:34.407562 26394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:34.407595 26396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:34.408614 26387 server_base.cc:1034] running on GCE node
I20260809 15:12:34.409387 26387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:34.410789 26387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:34.412024 26387 hybrid_clock.cc:648] HybridClock initialized: now 1786288354411984 us; error 38 us; skew 500 ppm
I20260809 15:12:34.412391 26387 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:12:34 dist-test-slave-nj21 krb5kdc[26362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288354, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:12:34.427446 26387 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 15:12:34.429018 26387 webserver.cc:460] Webserver started at http://127.0.0.1:36645/ using document root <none> and password file <none>
I20260809 15:12:34.429690 26387 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:34.429813 26387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:34.430124 26387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:34.433161 26387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "f109ffe1f4d94abeb8ce8fde75464087"
format_stamp: "Formatted at 2026-08-09 15:12:34 on dist-test-slave-nj21"
server_key: "d7be6730a9aaddd1218b0e797748afa9"
server_key_iv: "15388480850826a4e240e1afe67fd8fe"
server_key_version: "encryptionkey@0"
I20260809 15:12:34.433911 26387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "f109ffe1f4d94abeb8ce8fde75464087"
format_stamp: "Formatted at 2026-08-09 15:12:34 on dist-test-slave-nj21"
server_key: "d7be6730a9aaddd1218b0e797748afa9"
server_key_iv: "15388480850826a4e240e1afe67fd8fe"
server_key_version: "encryptionkey@0"
I20260809 15:12:34.438838 26387 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260809 15:12:34.442127 26403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:34.443349 26387 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:12:34.443501 26387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "f109ffe1f4d94abeb8ce8fde75464087"
format_stamp: "Formatted at 2026-08-09 15:12:34 on dist-test-slave-nj21"
server_key: "d7be6730a9aaddd1218b0e797748afa9"
server_key_iv: "15388480850826a4e240e1afe67fd8fe"
server_key_version: "encryptionkey@0"
I20260809 15:12:34.443707 26387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:34.459985 26387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:34.463914 26387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:34.464175 26387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:34.481709 26387 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35275
I20260809 15:12:34.481734 26484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35275 every 8 connection(s)
I20260809 15:12:34.483023 26387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:12:34.486601 26486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:34.487797 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 26387
I20260809 15:12:34.487986 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:12:34.488483 25311 external_mini_cluster.cc:1428] Setting key fd944d1a8380f7fb0ba124535d628583
I20260809 15:12:34.500562 26486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f109ffe1f4d94abeb8ce8fde75464087: Bootstrap starting.
Aug 09 15:12:34 dist-test-slave-nj21 krb5kdc[26362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 15:12:34.503443 26486 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f109ffe1f4d94abeb8ce8fde75464087: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:34.504522 26486 log.cc:824] T 00000000000000000000000000000000 P f109ffe1f4d94abeb8ce8fde75464087: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:34.507825 26486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f109ffe1f4d94abeb8ce8fde75464087: No bootstrap required, opened a new log
I20260809 15:12:34.512708 26490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:34.490576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.99:56761 (local address 127.0.0.1:35275)
0809 15:12:34.491444 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:34.491460 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:34.491513 (+ 53us) server_negotiation.cc:407] Connection header received
0809 15:12:34.493171 (+ 1658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:34.493205 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:34.493703 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:34.494121 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:34.495615 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:34.497630 (+ 2015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:34.498969 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:34.499997 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:34.500168 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:34.504457 (+ 4289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:34.504468 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:34.504485 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:12:34.504533 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:34.508298 (+ 3765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:34.509031 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:34.509040 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:34.509047 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:34.509261 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:34.509655 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:34.509660 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:34.509663 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:34.510350 (+ 687us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:34.510689 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:34.511053 (+ 364us) server_negotiation.cc:299] Negotiation successful
0809 15:12:34.511373 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":335,"threads_started":1}
I20260809 15:12:34.516507 25311 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:12:34.517666 26486 raft_consensus.cc:348] T 00000000000000000000000000000000 P f109ffe1f4d94abeb8ce8fde75464087 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f109ffe1f4d94abeb8ce8fde75464087" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35275 } }
I20260809 15:12:34.518078 26486 raft_consensus.cc:374] T 00000000000000000000000000000000 P f109ffe1f4d94abeb8ce8fde75464087 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:34.518167 26486 raft_consensus.cc:729] T 00000000000000000000000000000000 P f109ffe1f4d94abeb8ce8fde75464087 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f109ffe1f4d94abeb8ce8fde75464087, State: Initialized, Role: FOLLOWER
I20260809 15:12:34.518735 26486 consensus_queue.cc:260] T 00000000000000000000000000000000 P f109ffe1f4d94abeb8ce8fde75464087 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f109ffe1f4d94abeb8ce8fde75464087" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35275 } }
I20260809 15:12:34.519012 26486 raft_consensus.cc:388] T 00000000000000000000000000000000 P f109ffe1f4d94abeb8ce8fde75464087 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:12:34.519160 26486 raft_consensus.cc:482] T 00000000000000000000000000000000 P f109ffe1f4d94abeb8ce8fde75464087 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:12:34.519330 26486 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f109ffe1f4d94abeb8ce8fde75464087 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:34.522305 26486 raft_consensus.cc:504] T 00000000000000000000000000000000 P f109ffe1f4d94abeb8ce8fde75464087 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f109ffe1f4d94abeb8ce8fde75464087" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35275 } }
I20260809 15:12:34.522840 26486 leader_election.cc:302] T 00000000000000000000000000000000 P f109ffe1f4d94abeb8ce8fde75464087 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f109ffe1f4d94abeb8ce8fde75464087; no voters:
I20260809 15:12:34.523473 26486 leader_election.cc:288] T 00000000000000000000000000000000 P f109ffe1f4d94abeb8ce8fde75464087 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:12:34.523566 26499 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f109ffe1f4d94abeb8ce8fde75464087 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:12:34.525418 26486 sys_catalog.cc:564] T 00000000000000000000000000000000 P f109ffe1f4d94abeb8ce8fde75464087 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:12:34.530207 26499 raft_consensus.cc:686] T 00000000000000000000000000000000 P f109ffe1f4d94abeb8ce8fde75464087 [term 1 LEADER]: Becoming Leader. State: Replica: f109ffe1f4d94abeb8ce8fde75464087, State: Running, Role: LEADER
I20260809 15:12:34.530773 26499 consensus_queue.cc:237] T 00000000000000000000000000000000 P f109ffe1f4d94abeb8ce8fde75464087 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f109ffe1f4d94abeb8ce8fde75464087" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35275 } }
I20260809 15:12:34.533540 26503 sys_catalog.cc:455] T 00000000000000000000000000000000 P f109ffe1f4d94abeb8ce8fde75464087 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f109ffe1f4d94abeb8ce8fde75464087" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f109ffe1f4d94abeb8ce8fde75464087" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35275 } } }
I20260809 15:12:34.534075 26504 sys_catalog.cc:455] T 00000000000000000000000000000000 P f109ffe1f4d94abeb8ce8fde75464087 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f109ffe1f4d94abeb8ce8fde75464087. Latest consensus state: current_term: 1 leader_uuid: "f109ffe1f4d94abeb8ce8fde75464087" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f109ffe1f4d94abeb8ce8fde75464087" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35275 } } }
I20260809 15:12:34.534420 26504 sys_catalog.cc:458] T 00000000000000000000000000000000 P f109ffe1f4d94abeb8ce8fde75464087 [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:34.535229 26503 sys_catalog.cc:458] T 00000000000000000000000000000000 P f109ffe1f4d94abeb8ce8fde75464087 [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:34.540299 26490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:34.521697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.99:51239 (local address 127.0.0.1:35275)
0809 15:12:34.521871 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:34.521877 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:34.522348 (+ 471us) server_negotiation.cc:407] Connection header received
0809 15:12:34.522521 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:34.522527 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:34.522595 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:34.522670 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:34.524608 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:34.525849 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:34.527616 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:34.528426 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:34.528569 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:34.531900 (+ 3331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:34.531907 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:34.531913 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:12:34.531954 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:34.535962 (+ 4008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:34.537065 (+ 1103us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:34.537071 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:34.537075 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:34.537228 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:34.537835 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:34.537840 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:34.537844 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:34.538270 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:34.539544 (+ 1274us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:34.539744 (+ 200us) server_negotiation.cc:299] Negotiation successful
0809 15:12:34.539933 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 15:12:34.543427 26516 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:12:34.547749 26516 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:12:34.555974 26516 catalog_manager.cc:1347] Generated new cluster ID: 1cc09c8fef7145818f344e7638012c91
I20260809 15:12:34.556082 26516 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:12:34.564628 26516 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:12:34.566368 26516 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:12:34.577934 26516 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f109ffe1f4d94abeb8ce8fde75464087: Generated new TSK 0
I20260809 15:12:34.578956 26516 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:12:34.584460 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 26387
I20260809 15:12:34.586163 26479 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:12:34.675160 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 26387
W20260809 15:12:34.700156 26359 connection.cc:535] client connection to 127.0.0.1:35275 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35275): Connection reset by peer (error 104)
2026-08-09T15:12:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2178 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:12:34 dist-test-slave-nj21 krb5kdc[26531](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:12:34 dist-test-slave-nj21 krb5kdc[26531](info): set up 2 sockets
Aug 09 15:12:34 dist-test-slave-nj21 krb5kdc[26531](info): commencing operation
krb5kdc: starting...
W20260809 15:12:35.902463 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.002s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:12:36 dist-test-slave-nj21 krb5kdc[26531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:12:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:12:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 15:12:36.216655 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44391
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44391
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:12:36.556380 26549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:36.556638 26549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:36.556744 26549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:36.564227 26549 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:12:36.564337 26549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:36.564389 26549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:36.564448 26549 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:12:36.564489 26549 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:12:36.574127 26549 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:36.575281 26549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:36.576694 26549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:36.588155 26555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:36.588374 26556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:36.588783 26559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:36.589370 26549 server_base.cc:1034] running on GCE node
I20260809 15:12:36.590138 26549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:36.591432 26549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:36.592629 26549 hybrid_clock.cc:648] HybridClock initialized: now 1786288356592576 us; error 40 us; skew 500 ppm
I20260809 15:12:36.592964 26549 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:12:36 dist-test-slave-nj21 krb5kdc[26531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288356, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:12:36.596071 26549 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 15:12:36.597486 26549 webserver.cc:460] Webserver started at http://127.0.0.1:42513/ using document root <none> and password file <none>
I20260809 15:12:36.598104 26549 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:36.598204 26549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:36.598474 26549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:36.601464 26549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "392766f6a63643fb8f4a829861fea235"
format_stamp: "Formatted at 2026-08-09 15:12:36 on dist-test-slave-nj21"
server_key: "cbb3bbf2caf511dfbdf28a929a844f8b"
server_key_iv: "2005280c78bfaa5e9cac7dfc7cec6ae9"
server_key_version: "encryptionkey@0"
I20260809 15:12:36.602190 26549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "392766f6a63643fb8f4a829861fea235"
format_stamp: "Formatted at 2026-08-09 15:12:36 on dist-test-slave-nj21"
server_key: "cbb3bbf2caf511dfbdf28a929a844f8b"
server_key_iv: "2005280c78bfaa5e9cac7dfc7cec6ae9"
server_key_version: "encryptionkey@0"
I20260809 15:12:36.607246 26549 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 15:12:36.610605 26570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:36.611943 26549 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:12:36.612111 26549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "392766f6a63643fb8f4a829861fea235"
format_stamp: "Formatted at 2026-08-09 15:12:36 on dist-test-slave-nj21"
server_key: "cbb3bbf2caf511dfbdf28a929a844f8b"
server_key_iv: "2005280c78bfaa5e9cac7dfc7cec6ae9"
server_key_version: "encryptionkey@0"
I20260809 15:12:36.612289 26549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:36.632220 26549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:36.636034 26549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:36.636292 26549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:36.654073 26549 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44391
I20260809 15:12:36.654106 26641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44391 every 8 connection(s)
I20260809 15:12:36.655411 26549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:12:36.657074 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 26549
I20260809 15:12:36.657243 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:12:36.657678 25311 external_mini_cluster.cc:1428] Setting key e19991d8e0df3bf597d8a0b8b0ae65a1
I20260809 15:12:36.659411 26643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:36.672119 26647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:36.659429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.99:60541 (local address 127.0.0.1:44391)
0809 15:12:36.660242 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:36.660257 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:36.660310 (+ 53us) server_negotiation.cc:407] Connection header received
0809 15:12:36.662453 (+ 2143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:36.662481 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:36.662985 (+ 504us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:36.663401 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:36.663921 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:36.665901 (+ 1980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:36.667184 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:36.668191 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:36.668365 (+ 174us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:36.669423 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:36.669433 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:36.669465 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:36.670181 (+ 716us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:36.670269 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:36.670305 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:36.670469 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:36.670654 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 15:12:36.670814 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":329,"threads_started":1}
I20260809 15:12:36.673676 26643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 392766f6a63643fb8f4a829861fea235: Bootstrap starting.
I20260809 15:12:36.675652 25311 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:12:36.676962 26643 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 392766f6a63643fb8f4a829861fea235: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:36.678140 26643 log.cc:824] T 00000000000000000000000000000000 P 392766f6a63643fb8f4a829861fea235: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:36.680945 26643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 392766f6a63643fb8f4a829861fea235: No bootstrap required, opened a new log
I20260809 15:12:36.687110 26647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:36.680745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.99:49539 (local address 127.0.0.1:44391)
0809 15:12:36.680904 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:36.680910 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:36.681377 (+ 467us) server_negotiation.cc:407] Connection header received
0809 15:12:36.681503 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:36.681508 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:36.681579 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:36.681683 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:36.682141 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:36.683278 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:36.684603 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:36.685263 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:36.685405 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:36.686413 (+ 1008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:36.686419 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:36.686424 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:36.686444 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:36.686492 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:36.686513 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:36.686634 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:36.686731 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 15:12:36.686787 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 15:12:36.690117 26643 raft_consensus.cc:348] T 00000000000000000000000000000000 P 392766f6a63643fb8f4a829861fea235 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "392766f6a63643fb8f4a829861fea235" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44391 } }
I20260809 15:12:36.690464 26643 raft_consensus.cc:374] T 00000000000000000000000000000000 P 392766f6a63643fb8f4a829861fea235 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:36.690544 26643 raft_consensus.cc:729] T 00000000000000000000000000000000 P 392766f6a63643fb8f4a829861fea235 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 392766f6a63643fb8f4a829861fea235, State: Initialized, Role: FOLLOWER
I20260809 15:12:36.691079 26643 consensus_queue.cc:260] T 00000000000000000000000000000000 P 392766f6a63643fb8f4a829861fea235 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "392766f6a63643fb8f4a829861fea235" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44391 } }
I20260809 15:12:36.691283 26643 raft_consensus.cc:388] T 00000000000000000000000000000000 P 392766f6a63643fb8f4a829861fea235 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:12:36.691385 26643 raft_consensus.cc:482] T 00000000000000000000000000000000 P 392766f6a63643fb8f4a829861fea235 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:12:36.691512 26643 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 392766f6a63643fb8f4a829861fea235 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:36.693674 26643 raft_consensus.cc:504] T 00000000000000000000000000000000 P 392766f6a63643fb8f4a829861fea235 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "392766f6a63643fb8f4a829861fea235" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44391 } }
I20260809 15:12:36.694199 26643 leader_election.cc:302] T 00000000000000000000000000000000 P 392766f6a63643fb8f4a829861fea235 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 392766f6a63643fb8f4a829861fea235; no voters:
I20260809 15:12:36.694808 26643 leader_election.cc:288] T 00000000000000000000000000000000 P 392766f6a63643fb8f4a829861fea235 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:12:36.694922 26660 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 392766f6a63643fb8f4a829861fea235 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:12:36.695827 26660 raft_consensus.cc:686] T 00000000000000000000000000000000 P 392766f6a63643fb8f4a829861fea235 [term 1 LEADER]: Becoming Leader. State: Replica: 392766f6a63643fb8f4a829861fea235, State: Running, Role: LEADER
I20260809 15:12:36.696338 26660 consensus_queue.cc:237] T 00000000000000000000000000000000 P 392766f6a63643fb8f4a829861fea235 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "392766f6a63643fb8f4a829861fea235" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44391 } }
I20260809 15:12:36.697005 26643 sys_catalog.cc:564] T 00000000000000000000000000000000 P 392766f6a63643fb8f4a829861fea235 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:12:36.699306 26661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 392766f6a63643fb8f4a829861fea235 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 392766f6a63643fb8f4a829861fea235. Latest consensus state: current_term: 1 leader_uuid: "392766f6a63643fb8f4a829861fea235" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "392766f6a63643fb8f4a829861fea235" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44391 } } }
I20260809 15:12:36.699343 26663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 392766f6a63643fb8f4a829861fea235 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "392766f6a63643fb8f4a829861fea235" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "392766f6a63643fb8f4a829861fea235" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44391 } } }
I20260809 15:12:36.699748 26661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 392766f6a63643fb8f4a829861fea235 [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:36.699749 26663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 392766f6a63643fb8f4a829861fea235 [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:36.708215 26670 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:12:36.712903 26670 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:12:36.721748 26670 catalog_manager.cc:1347] Generated new cluster ID: 2db08e263c9343a6a1ce5e8db5fdd795
I20260809 15:12:36.721868 26670 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:12:36.733753 26670 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:12:36.734933 26670 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:12:36.742197 26670 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 392766f6a63643fb8f4a829861fea235: Generated new TSK 0
I20260809 15:12:36.742861 26670 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:12:36.757319 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 26549
W20260809 15:12:36.758446 26636 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.99:60541
W20260809 15:12:36.759167 25311 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 26549: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:12:36.759299 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 26549
W20260809 15:12:36.782675 26528 connection.cc:535] client connection to 127.0.0.1:44391 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44391): Connection reset by peer (error 104)
2026-08-09T15:12:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2082 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:12:36 dist-test-slave-nj21 krb5kdc[26690](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:12:36 dist-test-slave-nj21 krb5kdc[26690](info): set up 2 sockets
Aug 09 15:12:36 dist-test-slave-nj21 krb5kdc[26690](info): commencing operation
krb5kdc: starting...
W20260809 15:12:37.986913 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.001s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:12:38 dist-test-slave-nj21 krb5kdc[26690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:12:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:12:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 15:12:38.288197 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44971
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44971
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:12:38.628814 26709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:38.629113 26709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:38.629254 26709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:38.636595 26709 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:12:38.636726 26709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:38.636816 26709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:38.636896 26709 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:12:38.636955 26709 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:12:38.646855 26709 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44971
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:38.648259 26709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:38.649595 26709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:38.661094 26720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:38.661164 26715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:38.661164 26716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:38.662123 26709 server_base.cc:1034] running on GCE node
I20260809 15:12:38.662865 26709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:38.664263 26709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:38.665472 26709 hybrid_clock.cc:648] HybridClock initialized: now 1786288358665413 us; error 50 us; skew 500 ppm
I20260809 15:12:38.665799 26709 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:12:38 dist-test-slave-nj21 krb5kdc[26690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288358, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:12:38.668946 26709 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 15:12:38.670331 26709 webserver.cc:460] Webserver started at http://127.0.0.1:42667/ using document root <none> and password file <none>
I20260809 15:12:38.670975 26709 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:38.671126 26709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:38.671478 26709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:38.674592 26709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "65b2b25fd6f3447a84bf7cb98bff6b73"
format_stamp: "Formatted at 2026-08-09 15:12:38 on dist-test-slave-nj21"
server_key: "fe8ca0913453d86a7a9c83d657670529"
server_key_iv: "09126e79aa37f6d0130aed56e883576a"
server_key_version: "encryptionkey@0"
I20260809 15:12:38.675390 26709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "65b2b25fd6f3447a84bf7cb98bff6b73"
format_stamp: "Formatted at 2026-08-09 15:12:38 on dist-test-slave-nj21"
server_key: "fe8ca0913453d86a7a9c83d657670529"
server_key_iv: "09126e79aa37f6d0130aed56e883576a"
server_key_version: "encryptionkey@0"
I20260809 15:12:38.680720 26709 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 15:12:38.684147 26734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:38.685497 26709 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 15:12:38.685664 26709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "65b2b25fd6f3447a84bf7cb98bff6b73"
format_stamp: "Formatted at 2026-08-09 15:12:38 on dist-test-slave-nj21"
server_key: "fe8ca0913453d86a7a9c83d657670529"
server_key_iv: "09126e79aa37f6d0130aed56e883576a"
server_key_version: "encryptionkey@0"
I20260809 15:12:38.685848 26709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:38.691363 26709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:38.695158 26709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:38.695436 26709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:38.712944 26709 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44971
I20260809 15:12:38.712984 26797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44971 every 8 connection(s)
I20260809 15:12:38.714278 26709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:12:38.718091 26800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:38.718348 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 26709
I20260809 15:12:38.718537 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:12:38.719072 25311 external_mini_cluster.cc:1428] Setting key d4a68abb1e79f24050b6a9fc7d4d2f03
I20260809 15:12:38.726534 26804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:38.720853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.99:55561 (local address 127.0.0.1:44971)
0809 15:12:38.721715 (+ 862us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:38.721727 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:38.721767 (+ 40us) server_negotiation.cc:407] Connection header received
0809 15:12:38.723243 (+ 1476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:38.723264 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:38.723590 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:38.723996 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:38.725027 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:38.725034 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:38.725046 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:38.725060 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:38.725125 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:38.725145 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:38.725325 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:38.725438 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 15:12:38.725629 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":385,"threads_started":1}
I20260809 15:12:38.730355 25311 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:12:38.732506 26800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65b2b25fd6f3447a84bf7cb98bff6b73: Bootstrap starting.
I20260809 15:12:38.736482 26800 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65b2b25fd6f3447a84bf7cb98bff6b73: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:38.737540 26800 log.cc:824] T 00000000000000000000000000000000 P 65b2b25fd6f3447a84bf7cb98bff6b73: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:38.738227 26804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:38.735782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.99:46681 (local address 127.0.0.1:44971)
0809 15:12:38.735968 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:38.735974 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:38.736337 (+ 363us) server_negotiation.cc:407] Connection header received
0809 15:12:38.736515 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:38.736519 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:38.736575 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:38.736683 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:38.737562 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:38.737568 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:38.737572 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:38.737590 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:38.737634 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:38.737654 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:38.737752 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:38.737864 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 15:12:38.737923 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 15:12:38.740475 26800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65b2b25fd6f3447a84bf7cb98bff6b73: No bootstrap required, opened a new log
I20260809 15:12:38.747653 26800 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65b2b25fd6f3447a84bf7cb98bff6b73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65b2b25fd6f3447a84bf7cb98bff6b73" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44971 } }
I20260809 15:12:38.748049 26800 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65b2b25fd6f3447a84bf7cb98bff6b73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:38.748126 26800 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65b2b25fd6f3447a84bf7cb98bff6b73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65b2b25fd6f3447a84bf7cb98bff6b73, State: Initialized, Role: FOLLOWER
I20260809 15:12:38.748649 26800 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65b2b25fd6f3447a84bf7cb98bff6b73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65b2b25fd6f3447a84bf7cb98bff6b73" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44971 } }
I20260809 15:12:38.748870 26800 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65b2b25fd6f3447a84bf7cb98bff6b73 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:12:38.748972 26800 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65b2b25fd6f3447a84bf7cb98bff6b73 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:12:38.749094 26800 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65b2b25fd6f3447a84bf7cb98bff6b73 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:38.751410 26800 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65b2b25fd6f3447a84bf7cb98bff6b73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65b2b25fd6f3447a84bf7cb98bff6b73" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44971 } }
I20260809 15:12:38.751919 26800 leader_election.cc:302] T 00000000000000000000000000000000 P 65b2b25fd6f3447a84bf7cb98bff6b73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65b2b25fd6f3447a84bf7cb98bff6b73; no voters:
I20260809 15:12:38.752418 26800 leader_election.cc:288] T 00000000000000000000000000000000 P 65b2b25fd6f3447a84bf7cb98bff6b73 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:12:38.752574 26814 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65b2b25fd6f3447a84bf7cb98bff6b73 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:12:38.753273 26814 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65b2b25fd6f3447a84bf7cb98bff6b73 [term 1 LEADER]: Becoming Leader. State: Replica: 65b2b25fd6f3447a84bf7cb98bff6b73, State: Running, Role: LEADER
I20260809 15:12:38.753808 26814 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65b2b25fd6f3447a84bf7cb98bff6b73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65b2b25fd6f3447a84bf7cb98bff6b73" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44971 } }
I20260809 15:12:38.754419 26800 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65b2b25fd6f3447a84bf7cb98bff6b73 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:12:38.756479 26816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65b2b25fd6f3447a84bf7cb98bff6b73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65b2b25fd6f3447a84bf7cb98bff6b73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65b2b25fd6f3447a84bf7cb98bff6b73" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44971 } } }
I20260809 15:12:38.756834 26816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65b2b25fd6f3447a84bf7cb98bff6b73 [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:38.757299 26817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65b2b25fd6f3447a84bf7cb98bff6b73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65b2b25fd6f3447a84bf7cb98bff6b73. Latest consensus state: current_term: 1 leader_uuid: "65b2b25fd6f3447a84bf7cb98bff6b73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65b2b25fd6f3447a84bf7cb98bff6b73" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44971 } } }
I20260809 15:12:38.757591 26817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65b2b25fd6f3447a84bf7cb98bff6b73 [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:38.759191 26826 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:12:38.764608 26826 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:12:38.773620 26826 catalog_manager.cc:1347] Generated new cluster ID: cfe1e484311d42bdb6fd6b0fac942ed7
I20260809 15:12:38.773734 26826 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:12:38.787454 26826 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:12:38.787914 26826 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:12:38.810210 26826 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65b2b25fd6f3447a84bf7cb98bff6b73: Generated new TSK 0
I20260809 15:12:38.811267 26826 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:12:38.819828 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 26709
W20260809 15:12:38.821305 26790 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.99:55561
W20260809 15:12:38.822036 25311 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 26709: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:12:38.822202 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 26709
2026-08-09T15:12:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2066 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:12:38 dist-test-slave-nj21 krb5kdc[26847](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:12:38 dist-test-slave-nj21 krb5kdc[26847](info): set up 2 sockets
Aug 09 15:12:38 dist-test-slave-nj21 krb5kdc[26847](info): commencing operation
krb5kdc: starting...
W20260809 15:12:40.057536 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:12:40 dist-test-slave-nj21 krb5kdc[26847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:12:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:12:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 15:12:40.356360 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40839
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40839
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:12:40.693776 26867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:40.694057 26867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:40.694151 26867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:40.701258 26867 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:12:40.701364 26867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:40.701419 26867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:40.701479 26867 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:12:40.701524 26867 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:12:40.711190 26867 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40839
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:40.712433 26867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:40.713706 26867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:40.728120 26874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:40.728163 26879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:40.728204 26876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:40.728879 26867 server_base.cc:1034] running on GCE node
I20260809 15:12:40.729699 26867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:40.731115 26867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:40.732302 26867 hybrid_clock.cc:648] HybridClock initialized: now 1786288360732246 us; error 47 us; skew 500 ppm
I20260809 15:12:40.732635 26867 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:12:40 dist-test-slave-nj21 krb5kdc[26847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288360, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:12:40.738592 26867 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 15:12:40.740105 26867 webserver.cc:460] Webserver started at http://127.0.0.1:38567/ using document root <none> and password file <none>
I20260809 15:12:40.740770 26867 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:40.740883 26867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:40.741175 26867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:40.744206 26867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "e19484e0b6e24e47922c3cf3323aa610"
format_stamp: "Formatted at 2026-08-09 15:12:40 on dist-test-slave-nj21"
server_key: "bd1e6aa16fa5f0b19dde68704aa69c5d"
server_key_iv: "349b7437721fe577b97b030e7f3406c8"
server_key_version: "encryptionkey@0"
I20260809 15:12:40.744997 26867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "e19484e0b6e24e47922c3cf3323aa610"
format_stamp: "Formatted at 2026-08-09 15:12:40 on dist-test-slave-nj21"
server_key: "bd1e6aa16fa5f0b19dde68704aa69c5d"
server_key_iv: "349b7437721fe577b97b030e7f3406c8"
server_key_version: "encryptionkey@0"
I20260809 15:12:40.750247 26867 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 15:12:40.753603 26887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:40.754904 26867 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 15:12:40.755064 26867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "e19484e0b6e24e47922c3cf3323aa610"
format_stamp: "Formatted at 2026-08-09 15:12:40 on dist-test-slave-nj21"
server_key: "bd1e6aa16fa5f0b19dde68704aa69c5d"
server_key_iv: "349b7437721fe577b97b030e7f3406c8"
server_key_version: "encryptionkey@0"
I20260809 15:12:40.755239 26867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:40.770423 26867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:40.774791 26867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:40.775071 26867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:40.792639 26867 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40839
I20260809 15:12:40.792667 26954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40839 every 8 connection(s)
I20260809 15:12:40.794030 26867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:12:40.797719 26955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:40.801854 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 26867
I20260809 15:12:40.802032 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:12:40.802493 25311 external_mini_cluster.cc:1428] Setting key 9734408b458fda9bb7f4425a608cb677
I20260809 15:12:40.809731 26955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19484e0b6e24e47922c3cf3323aa610: Bootstrap starting.
I20260809 15:12:40.813333 26955 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e19484e0b6e24e47922c3cf3323aa610: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:40.814436 26955 log.cc:824] T 00000000000000000000000000000000 P e19484e0b6e24e47922c3cf3323aa610: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:40.817446 26955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19484e0b6e24e47922c3cf3323aa610: No bootstrap required, opened a new log
Aug 09 15:12:40 dist-test-slave-nj21 krb5kdc[26847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 15:12:40.826452 26955 raft_consensus.cc:348] T 00000000000000000000000000000000 P e19484e0b6e24e47922c3cf3323aa610 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19484e0b6e24e47922c3cf3323aa610" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40839 } }
I20260809 15:12:40.826798 26955 raft_consensus.cc:374] T 00000000000000000000000000000000 P e19484e0b6e24e47922c3cf3323aa610 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:40.826941 26955 raft_consensus.cc:729] T 00000000000000000000000000000000 P e19484e0b6e24e47922c3cf3323aa610 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e19484e0b6e24e47922c3cf3323aa610, State: Initialized, Role: FOLLOWER
I20260809 15:12:40.827612 26955 consensus_queue.cc:260] T 00000000000000000000000000000000 P e19484e0b6e24e47922c3cf3323aa610 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19484e0b6e24e47922c3cf3323aa610" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40839 } }
I20260809 15:12:40.827916 26955 raft_consensus.cc:388] T 00000000000000000000000000000000 P e19484e0b6e24e47922c3cf3323aa610 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:12:40.828076 26955 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19484e0b6e24e47922c3cf3323aa610 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:12:40.828244 26955 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e19484e0b6e24e47922c3cf3323aa610 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:40.828809 26959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:40.804380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.99:44405 (local address 127.0.0.1:40839)
0809 15:12:40.805261 (+ 881us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:40.805277 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:40.805329 (+ 52us) server_negotiation.cc:407] Connection header received
0809 15:12:40.808017 (+ 2688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:40.808047 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:40.808524 (+ 477us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:40.808940 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:40.810577 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:40.812603 (+ 2026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:40.814011 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:40.815071 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:40.815233 (+ 162us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:40.819900 (+ 4667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:40.819915 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:40.819935 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:12:40.819982 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:40.824068 (+ 4086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:40.824961 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:40.824969 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:40.824975 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:40.825182 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:40.825709 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:40.825713 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:40.825716 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:40.826296 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:40.826633 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:40.827235 (+ 602us) server_negotiation.cc:299] Negotiation successful
0809 15:12:40.827545 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":355,"threads_started":1}
I20260809 15:12:40.831039 26955 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19484e0b6e24e47922c3cf3323aa610 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19484e0b6e24e47922c3cf3323aa610" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40839 } }
I20260809 15:12:40.831573 26955 leader_election.cc:302] T 00000000000000000000000000000000 P e19484e0b6e24e47922c3cf3323aa610 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e19484e0b6e24e47922c3cf3323aa610; no voters:
I20260809 15:12:40.832229 26955 leader_election.cc:288] T 00000000000000000000000000000000 P e19484e0b6e24e47922c3cf3323aa610 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:12:40.832347 26962 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e19484e0b6e24e47922c3cf3323aa610 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:12:40.832979 25311 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:12:40.834450 26955 sys_catalog.cc:564] T 00000000000000000000000000000000 P e19484e0b6e24e47922c3cf3323aa610 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:12:40.840207 26962 raft_consensus.cc:686] T 00000000000000000000000000000000 P e19484e0b6e24e47922c3cf3323aa610 [term 1 LEADER]: Becoming Leader. State: Replica: e19484e0b6e24e47922c3cf3323aa610, State: Running, Role: LEADER
I20260809 15:12:40.840930 26962 consensus_queue.cc:237] T 00000000000000000000000000000000 P e19484e0b6e24e47922c3cf3323aa610 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19484e0b6e24e47922c3cf3323aa610" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40839 } }
I20260809 15:12:40.844292 26964 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19484e0b6e24e47922c3cf3323aa610 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e19484e0b6e24e47922c3cf3323aa610" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19484e0b6e24e47922c3cf3323aa610" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40839 } } }
I20260809 15:12:40.844696 26964 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19484e0b6e24e47922c3cf3323aa610 [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:40.845206 26965 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19484e0b6e24e47922c3cf3323aa610 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e19484e0b6e24e47922c3cf3323aa610. Latest consensus state: current_term: 1 leader_uuid: "e19484e0b6e24e47922c3cf3323aa610" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19484e0b6e24e47922c3cf3323aa610" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40839 } } }
I20260809 15:12:40.845515 26965 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19484e0b6e24e47922c3cf3323aa610 [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:40.860947 26981 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:12:40.865595 26959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:40.844688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.99:44339 (local address 127.0.0.1:40839)
0809 15:12:40.844879 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:40.844885 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:40.846330 (+ 1445us) server_negotiation.cc:407] Connection header received
0809 15:12:40.847105 (+ 775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:40.847111 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:40.847204 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:40.847312 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:40.849684 (+ 2372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:40.850933 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:40.852872 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:40.853588 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:40.853682 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:40.857637 (+ 3955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:40.857644 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:40.857650 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:12:40.857704 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:40.861411 (+ 3707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:40.862335 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:40.862341 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:40.862344 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:40.862511 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:40.862969 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:40.862972 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:40.862974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:40.863423 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:40.863765 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:40.865010 (+ 1245us) server_negotiation.cc:299] Negotiation successful
0809 15:12:40.865210 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 15:12:40.866830 26981 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:12:40.875422 26981 catalog_manager.cc:1347] Generated new cluster ID: 4b9dfe049fdc42baa8c714796ec59973
I20260809 15:12:40.875521 26981 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:12:40.909899 26981 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:12:40.911146 26981 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:12:40.924019 26981 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e19484e0b6e24e47922c3cf3323aa610: Generated new TSK 0
I20260809 15:12:40.924801 26981 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:12:40.936690 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 26867
I20260809 15:12:40.938007 26949 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:12:41.031489 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 26867
W20260809 15:12:41.054678 26844 connection.cc:535] client connection to 127.0.0.1:40839 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40839): Connection reset by peer (error 104)
2026-08-09T15:12:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2205 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:12:41 dist-test-slave-nj21 krb5kdc[27003](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:12:41 dist-test-slave-nj21 krb5kdc[27003](info): set up 2 sockets
Aug 09 15:12:41 dist-test-slave-nj21 krb5kdc[27003](info): commencing operation
krb5kdc: starting...
W20260809 15:12:42.272818 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.001s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:12:42 dist-test-slave-nj21 krb5kdc[27003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:12:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:12:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 15:12:42.576483 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41545
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:12:42.918181 27022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:42.918455 27022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:42.918586 27022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:42.925684 27022 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:12:42.925832 27022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:42.925908 27022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:42.925999 27022 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:12:42.926056 27022 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:12:42.935757 27022 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41545
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:42.937093 27022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:42.938378 27022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:42.949905 27029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:42.949946 27031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:42.949976 27035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:42.951154 27022 server_base.cc:1034] running on GCE node
I20260809 15:12:42.951944 27022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:42.953327 27022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:42.954527 27022 hybrid_clock.cc:648] HybridClock initialized: now 1786288362954472 us; error 50 us; skew 500 ppm
I20260809 15:12:42.954840 27022 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:12:42 dist-test-slave-nj21 krb5kdc[27003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288362, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:12:42.957912 27022 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 15:12:42.959416 27022 webserver.cc:460] Webserver started at http://127.0.0.1:41981/ using document root <none> and password file <none>
I20260809 15:12:42.960132 27022 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:42.960250 27022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:42.960544 27022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:42.963523 27022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "3da46a44660c48799a991dd7666f2a08"
format_stamp: "Formatted at 2026-08-09 15:12:42 on dist-test-slave-nj21"
server_key: "af6e2afc0cd255868608886564f89f37"
server_key_iv: "0042b918dde678c5c5c970ecdec42d49"
server_key_version: "encryptionkey@0"
I20260809 15:12:42.964294 27022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "3da46a44660c48799a991dd7666f2a08"
format_stamp: "Formatted at 2026-08-09 15:12:42 on dist-test-slave-nj21"
server_key: "af6e2afc0cd255868608886564f89f37"
server_key_iv: "0042b918dde678c5c5c970ecdec42d49"
server_key_version: "encryptionkey@0"
I20260809 15:12:42.969312 27022 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260809 15:12:42.972606 27044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:42.973835 27022 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260809 15:12:42.973999 27022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "3da46a44660c48799a991dd7666f2a08"
format_stamp: "Formatted at 2026-08-09 15:12:42 on dist-test-slave-nj21"
server_key: "af6e2afc0cd255868608886564f89f37"
server_key_iv: "0042b918dde678c5c5c970ecdec42d49"
server_key_version: "encryptionkey@0"
I20260809 15:12:42.974172 27022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:42.990029 27022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:42.993743 27022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:42.994001 27022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:43.011780 27022 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41545
I20260809 15:12:43.011798 27120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41545 every 8 connection(s)
I20260809 15:12:43.013128 27022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:12:43.016800 27122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:43.017309 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 27022
I20260809 15:12:43.017518 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:12:43.018169 25311 external_mini_cluster.cc:1428] Setting key 854400d626f87facac22a24f4ed2b51d
I20260809 15:12:43.030463 27122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3da46a44660c48799a991dd7666f2a08: Bootstrap starting.
I20260809 15:12:43.031841 27125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:43.020013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.99:54803 (local address 127.0.0.1:41545)
0809 15:12:43.020773 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:43.020785 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:43.020829 (+ 44us) server_negotiation.cc:407] Connection header received
0809 15:12:43.023120 (+ 2291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:43.023144 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:43.023538 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:43.023910 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:43.024493 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:43.025700 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:43.027578 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:43.028287 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:43.028419 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:43.029637 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:43.029644 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:43.029656 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:43.030160 (+ 504us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:43.030247 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:43.030271 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:43.030386 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:43.030682 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 15:12:43.030863 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":355,"threads_started":1}
I20260809 15:12:43.033679 27122 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3da46a44660c48799a991dd7666f2a08: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:43.034885 27122 log.cc:824] T 00000000000000000000000000000000 P 3da46a44660c48799a991dd7666f2a08: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:43.035367 25311 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:12:43.037612 27122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3da46a44660c48799a991dd7666f2a08: No bootstrap required, opened a new log
I20260809 15:12:43.046433 27125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:43.039909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.99:37957 (local address 127.0.0.1:41545)
0809 15:12:43.040099 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:43.040105 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:43.040645 (+ 540us) server_negotiation.cc:407] Connection header received
0809 15:12:43.040795 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:43.040802 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:43.040863 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:43.040946 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:43.041529 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:43.042269 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:43.044078 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:43.044519 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:43.044622 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:43.045747 (+ 1125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:43.045751 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:43.045754 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:43.045770 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:43.045820 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:43.045834 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:43.045916 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:43.046096 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 15:12:43.046137 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 15:12:43.046959 27122 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3da46a44660c48799a991dd7666f2a08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3da46a44660c48799a991dd7666f2a08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41545 } }
I20260809 15:12:43.047336 27122 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3da46a44660c48799a991dd7666f2a08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:43.047431 27122 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3da46a44660c48799a991dd7666f2a08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da46a44660c48799a991dd7666f2a08, State: Initialized, Role: FOLLOWER
I20260809 15:12:43.048026 27122 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3da46a44660c48799a991dd7666f2a08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3da46a44660c48799a991dd7666f2a08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41545 } }
I20260809 15:12:43.048267 27122 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3da46a44660c48799a991dd7666f2a08 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:12:43.048380 27122 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3da46a44660c48799a991dd7666f2a08 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:12:43.048516 27122 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3da46a44660c48799a991dd7666f2a08 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:43.051132 27122 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3da46a44660c48799a991dd7666f2a08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3da46a44660c48799a991dd7666f2a08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41545 } }
I20260809 15:12:43.051581 27122 leader_election.cc:302] T 00000000000000000000000000000000 P 3da46a44660c48799a991dd7666f2a08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3da46a44660c48799a991dd7666f2a08; no voters:
I20260809 15:12:43.052153 27122 leader_election.cc:288] T 00000000000000000000000000000000 P 3da46a44660c48799a991dd7666f2a08 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:12:43.052255 27140 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3da46a44660c48799a991dd7666f2a08 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:12:43.052886 27140 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3da46a44660c48799a991dd7666f2a08 [term 1 LEADER]: Becoming Leader. State: Replica: 3da46a44660c48799a991dd7666f2a08, State: Running, Role: LEADER
I20260809 15:12:43.053401 27140 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3da46a44660c48799a991dd7666f2a08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3da46a44660c48799a991dd7666f2a08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41545 } }
I20260809 15:12:43.054116 27122 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3da46a44660c48799a991dd7666f2a08 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:12:43.055887 27144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3da46a44660c48799a991dd7666f2a08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3da46a44660c48799a991dd7666f2a08. Latest consensus state: current_term: 1 leader_uuid: "3da46a44660c48799a991dd7666f2a08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3da46a44660c48799a991dd7666f2a08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41545 } } }
I20260809 15:12:43.056254 27144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3da46a44660c48799a991dd7666f2a08 [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:43.057605 27142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3da46a44660c48799a991dd7666f2a08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3da46a44660c48799a991dd7666f2a08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3da46a44660c48799a991dd7666f2a08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41545 } } }
I20260809 15:12:43.057832 27142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3da46a44660c48799a991dd7666f2a08 [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:43.058290 27148 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:12:43.063015 27148 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:12:43.071578 27148 catalog_manager.cc:1347] Generated new cluster ID: 57e4286c99c54b8bbb160c9178591ed0
I20260809 15:12:43.071703 27148 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:12:43.083000 27148 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:12:43.084501 27148 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:12:43.096856 27148 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3da46a44660c48799a991dd7666f2a08: Generated new TSK 0
I20260809 15:12:43.097638 27148 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:12:43.107005 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 27022
W20260809 15:12:43.108180 27115 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.99:54803
W20260809 15:12:43.108888 25311 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 27022: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:12:43.109072 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 27022
W20260809 15:12:43.127265 26999 connection.cc:535] client connection to 127.0.0.1:41545 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41545): Connection reset by peer (error 104)
2026-08-09T15:12:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2072 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:12:43 dist-test-slave-nj21 krb5kdc[27177](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:12:43 dist-test-slave-nj21 krb5kdc[27177](info): set up 2 sockets
Aug 09 15:12:43 dist-test-slave-nj21 krb5kdc[27177](info): commencing operation
krb5kdc: starting...
W20260809 15:12:44.350843 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.001s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:12:44 dist-test-slave-nj21 krb5kdc[27177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:12:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:12:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 15:12:44.646395 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45055
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45055
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:12:44.987169 27191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:44.987440 27191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:44.987569 27191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:44.994766 27191 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:12:44.994871 27191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:44.994937 27191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:44.995020 27191 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:12:44.995081 27191 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:12:45.005295 27191 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45055
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288021313853-25311-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:45.006615 27191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:45.008072 27191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:45.019701 27199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:45.019644 27204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:45.019915 27198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:45.020546 27191 server_base.cc:1034] running on GCE node
I20260809 15:12:45.021230 27191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:45.022517 27191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:45.023746 27191 hybrid_clock.cc:648] HybridClock initialized: now 1786288365023700 us; error 28 us; skew 500 ppm
I20260809 15:12:45.024085 27191 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:12:45 dist-test-slave-nj21 krb5kdc[27177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288365, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:12:45.027136 27191 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 15:12:45.028566 27191 webserver.cc:460] Webserver started at http://127.0.0.1:39243/ using document root <none> and password file <none>
I20260809 15:12:45.029217 27191 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:45.029337 27191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:45.029647 27191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:45.032658 27191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "49ba7322822f42918c4d049046304930"
format_stamp: "Formatted at 2026-08-09 15:12:45 on dist-test-slave-nj21"
server_key: "6c2d68298600f054c0731183447dff32"
server_key_iv: "7b9c165386c652d478031e6de6e9e732"
server_key_version: "encryptionkey@0"
I20260809 15:12:45.033413 27191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "49ba7322822f42918c4d049046304930"
format_stamp: "Formatted at 2026-08-09 15:12:45 on dist-test-slave-nj21"
server_key: "6c2d68298600f054c0731183447dff32"
server_key_iv: "7b9c165386c652d478031e6de6e9e732"
server_key_version: "encryptionkey@0"
I20260809 15:12:45.038475 27191 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 15:12:45.041854 27214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:45.043143 27191 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:12:45.043308 27191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "49ba7322822f42918c4d049046304930"
format_stamp: "Formatted at 2026-08-09 15:12:45 on dist-test-slave-nj21"
server_key: "6c2d68298600f054c0731183447dff32"
server_key_iv: "7b9c165386c652d478031e6de6e9e732"
server_key_version: "encryptionkey@0"
I20260809 15:12:45.043484 27191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:45.049064 27191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:45.053130 27191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:45.053406 27191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:45.071084 27191 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45055
I20260809 15:12:45.071123 27293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45055 every 8 connection(s)
I20260809 15:12:45.072461 27191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:12:45.075565 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 27191
I20260809 15:12:45.075798 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:12:45.076392 25311 external_mini_cluster.cc:1428] Setting key 46074203ac2ada7eea593ba96e57d518
I20260809 15:12:45.076439 27294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:45.089738 27297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:45.084008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.99:50829 (local address 127.0.0.1:45055)
0809 15:12:45.084753 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:45.084765 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:45.084805 (+ 40us) server_negotiation.cc:407] Connection header received
0809 15:12:45.086438 (+ 1633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:45.086461 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:45.086796 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:45.087148 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:45.088183 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:45.088188 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:45.088204 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:45.088218 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:45.088272 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:45.088290 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:45.088496 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:45.088623 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 15:12:45.088804 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":328,"threads_started":1}
I20260809 15:12:45.090055 27294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49ba7322822f42918c4d049046304930: Bootstrap starting.
I20260809 15:12:45.093166 27294 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49ba7322822f42918c4d049046304930: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:45.093180 25311 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:12:45.094120 27294 log.cc:824] T 00000000000000000000000000000000 P 49ba7322822f42918c4d049046304930: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:45.096782 27294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49ba7322822f42918c4d049046304930: No bootstrap required, opened a new log
I20260809 15:12:45.099730 27297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:45.097287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.99:43525 (local address 127.0.0.1:45055)
0809 15:12:45.097420 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:45.097423 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:45.097942 (+ 519us) server_negotiation.cc:407] Connection header received
0809 15:12:45.098092 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:45.098095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:45.098139 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:45.098214 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:45.099050 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:45.099056 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:45.099061 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:45.099083 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:45.099121 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:45.099135 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:45.099243 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:45.099370 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 15:12:45.099435 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 15:12:45.104580 27294 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49ba7322822f42918c4d049046304930 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49ba7322822f42918c4d049046304930" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45055 } }
I20260809 15:12:45.104939 27294 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49ba7322822f42918c4d049046304930 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:45.105029 27294 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49ba7322822f42918c4d049046304930 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49ba7322822f42918c4d049046304930, State: Initialized, Role: FOLLOWER
I20260809 15:12:45.105543 27294 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49ba7322822f42918c4d049046304930 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49ba7322822f42918c4d049046304930" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45055 } }
I20260809 15:12:45.105784 27294 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49ba7322822f42918c4d049046304930 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:12:45.105911 27294 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49ba7322822f42918c4d049046304930 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:12:45.106055 27294 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49ba7322822f42918c4d049046304930 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:45.108395 27294 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49ba7322822f42918c4d049046304930 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49ba7322822f42918c4d049046304930" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45055 } }
I20260809 15:12:45.108853 27294 leader_election.cc:302] T 00000000000000000000000000000000 P 49ba7322822f42918c4d049046304930 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49ba7322822f42918c4d049046304930; no voters:
I20260809 15:12:45.109406 27294 leader_election.cc:288] T 00000000000000000000000000000000 P 49ba7322822f42918c4d049046304930 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:12:45.109548 27309 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49ba7322822f42918c4d049046304930 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:12:45.110095 27309 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49ba7322822f42918c4d049046304930 [term 1 LEADER]: Becoming Leader. State: Replica: 49ba7322822f42918c4d049046304930, State: Running, Role: LEADER
I20260809 15:12:45.110677 27309 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49ba7322822f42918c4d049046304930 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49ba7322822f42918c4d049046304930" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45055 } }
I20260809 15:12:45.111853 27294 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49ba7322822f42918c4d049046304930 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:12:45.113283 27313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49ba7322822f42918c4d049046304930 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49ba7322822f42918c4d049046304930" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49ba7322822f42918c4d049046304930" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45055 } } }
I20260809 15:12:45.113758 27313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49ba7322822f42918c4d049046304930 [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:45.113876 27312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49ba7322822f42918c4d049046304930 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49ba7322822f42918c4d049046304930. Latest consensus state: current_term: 1 leader_uuid: "49ba7322822f42918c4d049046304930" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49ba7322822f42918c4d049046304930" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45055 } } }
I20260809 15:12:45.114219 27312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49ba7322822f42918c4d049046304930 [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:45.115317 27318 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:12:45.120491 27318 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:12:45.128942 27318 catalog_manager.cc:1347] Generated new cluster ID: e24b5f231bdf418a9cfbbe6cfb5c0234
I20260809 15:12:45.129089 27318 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:12:45.148124 27318 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:12:45.148648 27318 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:12:45.164345 27318 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49ba7322822f42918c4d049046304930: Generated new TSK 0
I20260809 15:12:45.165201 27318 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:12:45.180826 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 27191
W20260809 15:12:45.182168 27287 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.99:50829
W20260809 15:12:45.182874 25311 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 27191: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:12:45.183007 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 27191
2026-08-09T15:12:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2079 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:12:45 dist-test-slave-nj21 krb5kdc[27344](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:12:45 dist-test-slave-nj21 krb5kdc[27344](info): set up 2 sockets
Aug 09 15:12:45 dist-test-slave-nj21 krb5kdc[27344](info): commencing operation
krb5kdc: starting...
W20260809 15:12:46.435758 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.007s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:12:46 dist-test-slave-nj21 krb5kdc[27344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:12:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:12:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
I20260809 15:12:46.730207 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:44623
--webserver_interface=127.24.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:45081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:44623
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:12:47.073446 27365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:47.073748 27365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:47.073868 27365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:47.081211 27365 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:12:47.081337 27365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:47.081393 27365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:47.081454 27365 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:12:47.081497 27365 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:12:47.091517 27365 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:45081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:44623
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:44623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:47.092875 27365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:47.094264 27365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:47.105983 27373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:47.106015 27375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:47.105938 27377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:47.107033 27365 server_base.cc:1034] running on GCE node
I20260809 15:12:47.107906 27365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:47.109299 27365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:47.110513 27365 hybrid_clock.cc:648] HybridClock initialized: now 1786288367110457 us; error 38 us; skew 500 ppm
I20260809 15:12:47.110875 27365 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:12:47 dist-test-slave-nj21 krb5kdc[27344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288367, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:12:47.114068 27365 init.cc:377] Logged in from keytab as kudu/127.24.183.254@KRBTEST.COM (short username kudu)
I20260809 15:12:47.115571 27365 webserver.cc:460] Webserver started at http://127.24.183.254:38881/ using document root <none> and password file <none>
I20260809 15:12:47.116276 27365 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:47.116415 27365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:47.116737 27365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:47.119804 27365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "46a112df7a864efd9dc40891956e5070"
format_stamp: "Formatted at 2026-08-09 15:12:47 on dist-test-slave-nj21"
server_key: "b71599364a511e394e29c572eb46e61d"
server_key_iv: "98d069e85cdbb197de01be79e2167bb9"
server_key_version: "encryptionkey@0"
I20260809 15:12:47.120571 27365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "46a112df7a864efd9dc40891956e5070"
format_stamp: "Formatted at 2026-08-09 15:12:47 on dist-test-slave-nj21"
server_key: "b71599364a511e394e29c572eb46e61d"
server_key_iv: "98d069e85cdbb197de01be79e2167bb9"
server_key_version: "encryptionkey@0"
I20260809 15:12:47.125514 27365 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 15:12:47.128831 27387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:47.130084 27365 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:12:47.130259 27365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "46a112df7a864efd9dc40891956e5070"
format_stamp: "Formatted at 2026-08-09 15:12:47 on dist-test-slave-nj21"
server_key: "b71599364a511e394e29c572eb46e61d"
server_key_iv: "98d069e85cdbb197de01be79e2167bb9"
server_key_version: "encryptionkey@0"
I20260809 15:12:47.130436 27365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:47.153465 27365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:47.157872 27365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:47.158155 27365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:47.176239 27365 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:44623
I20260809 15:12:47.176263 27457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:44623 every 8 connection(s)
I20260809 15:12:47.177609 27365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:12:47.177944 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 27365
I20260809 15:12:47.178102 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:12:47.178594 25311 external_mini_cluster.cc:1428] Setting key 9d3fb31c607b34136403ef58c16ccc37
I20260809 15:12:47.181977 27458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 15:12:47 dist-test-slave-nj21 krb5kdc[27344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:12:47.192828 27458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46a112df7a864efd9dc40891956e5070: Bootstrap starting.
I20260809 15:12:47.196445 27458 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46a112df7a864efd9dc40891956e5070: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:47.197682 27458 log.cc:824] T 00000000000000000000000000000000 P 46a112df7a864efd9dc40891956e5070: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:47.201092 27458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46a112df7a864efd9dc40891956e5070: No bootstrap required, opened a new log
I20260809 15:12:47.201236 27462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:47.180446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37585 (local address 127.24.183.254:44623)
0809 15:12:47.181200 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:47.181212 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:47.181252 (+ 40us) server_negotiation.cc:407] Connection header received
0809 15:12:47.182926 (+ 1674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:47.182948 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:47.183336 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:47.183752 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:47.185289 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:47.186624 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:47.187905 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:47.188566 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:47.188724 (+ 158us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:47.193126 (+ 4402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:47.193134 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:47.193147 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:12:47.193183 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:47.196518 (+ 3335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:47.197451 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:47.197466 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:47.197472 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:47.197674 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:47.198267 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:47.198273 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:47.198276 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:47.199056 (+ 780us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:47.199329 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:47.199911 (+ 582us) server_negotiation.cc:299] Negotiation successful
0809 15:12:47.200224 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":338,"threads_started":1}
I20260809 15:12:47.205103 25311 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:12:47.209303 27458 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46a112df7a864efd9dc40891956e5070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a112df7a864efd9dc40891956e5070" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 44623 } }
I20260809 15:12:47.209774 27458 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46a112df7a864efd9dc40891956e5070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:47.209892 27458 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46a112df7a864efd9dc40891956e5070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a112df7a864efd9dc40891956e5070, State: Initialized, Role: FOLLOWER
I20260809 15:12:47.210512 27458 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46a112df7a864efd9dc40891956e5070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a112df7a864efd9dc40891956e5070" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 44623 } }
I20260809 15:12:47.210807 27458 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46a112df7a864efd9dc40891956e5070 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:12:47.210955 27458 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46a112df7a864efd9dc40891956e5070 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:12:47.211151 27458 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46a112df7a864efd9dc40891956e5070 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:47.214056 27458 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46a112df7a864efd9dc40891956e5070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a112df7a864efd9dc40891956e5070" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 44623 } }
I20260809 15:12:47.214612 27458 leader_election.cc:302] T 00000000000000000000000000000000 P 46a112df7a864efd9dc40891956e5070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46a112df7a864efd9dc40891956e5070; no voters:
I20260809 15:12:47.215263 27458 leader_election.cc:288] T 00000000000000000000000000000000 P 46a112df7a864efd9dc40891956e5070 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:12:47.215401 27470 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46a112df7a864efd9dc40891956e5070 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:12:47.217365 27458 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46a112df7a864efd9dc40891956e5070 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:12:47.220873 27470 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46a112df7a864efd9dc40891956e5070 [term 1 LEADER]: Becoming Leader. State: Replica: 46a112df7a864efd9dc40891956e5070, State: Running, Role: LEADER
I20260809 15:12:47.221513 27470 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46a112df7a864efd9dc40891956e5070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a112df7a864efd9dc40891956e5070" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 44623 } }
I20260809 15:12:47.223507 27472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46a112df7a864efd9dc40891956e5070 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46a112df7a864efd9dc40891956e5070. Latest consensus state: current_term: 1 leader_uuid: "46a112df7a864efd9dc40891956e5070" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a112df7a864efd9dc40891956e5070" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 44623 } } }
I20260809 15:12:47.223907 27472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46a112df7a864efd9dc40891956e5070 [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:47.224344 27471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46a112df7a864efd9dc40891956e5070 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46a112df7a864efd9dc40891956e5070" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a112df7a864efd9dc40891956e5070" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 44623 } } }
I20260809 15:12:47.224637 27471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46a112df7a864efd9dc40891956e5070 [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:47.226862 27462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:47.209179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49891 (local address 127.24.183.254:44623)
0809 15:12:47.209316 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:47.209320 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:47.209853 (+ 533us) server_negotiation.cc:407] Connection header received
0809 15:12:47.210036 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:47.210040 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:47.210095 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:47.210178 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:47.212277 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:47.213060 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:47.214692 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:47.215270 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:47.215394 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:47.218729 (+ 3335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:47.218736 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:47.218742 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:12:47.218787 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:47.222501 (+ 3714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:47.223399 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:47.223405 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:47.223408 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:47.223588 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:47.224201 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:47.224207 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:47.224210 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:47.224659 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:47.224982 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:47.226242 (+ 1260us) server_negotiation.cc:299] Negotiation successful
0809 15:12:47.226457 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 15:12:47.228580 27483 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:12:47.233443 27483 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:12:47.242055 27483 catalog_manager.cc:1347] Generated new cluster ID: ef1fe2165de849d083b62d6fd902615d
I20260809 15:12:47.242162 27483 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:12:47.264550 27483 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:12:47.265812 27483 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:12:47.281260 27483 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46a112df7a864efd9dc40891956e5070: Generated new TSK 0
I20260809 15:12:47.282070 27483 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:12:47.288920 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 27365
I20260809 15:12:47.290416 27451 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:12:47.383436 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 27365
W20260809 15:12:47.408596 27340 connection.cc:535] client connection to 127.24.183.254:44623 recv error: Network error: failed to read from TLS socket (remote: 127.24.183.254:44623): Connection reset by peer (error 104)
2026-08-09T15:12:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2201 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:12:47 dist-test-slave-nj21 krb5kdc[27498](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:12:47 dist-test-slave-nj21 krb5kdc[27498](info): set up 2 sockets
Aug 09 15:12:47 dist-test-slave-nj21 krb5kdc[27498](info): commencing operation
krb5kdc: starting...
W20260809 15:12:48.619766 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:12:48 dist-test-slave-nj21 krb5kdc[27498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:12:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:12:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
I20260809 15:12:48.918219 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:34781
--webserver_interface=127.24.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:43355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:34781
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:12:49.260267 27513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:49.260564 27513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:49.260705 27513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:49.267968 27513 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:12:49.268085 27513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:49.268162 27513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:49.268245 27513 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:12:49.268306 27513 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:12:49.278292 27513 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:43355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:34781
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:34781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:49.280087 27513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:49.281620 27513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:49.293164 27526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:49.293200 27532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:49.293267 27525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:49.293838 27513 server_base.cc:1034] running on GCE node
I20260809 15:12:49.294634 27513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:49.296125 27513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:49.297315 27513 hybrid_clock.cc:648] HybridClock initialized: now 1786288369297281 us; error 47 us; skew 500 ppm
I20260809 15:12:49.297649 27513 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:12:49 dist-test-slave-nj21 krb5kdc[27498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288369, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:12:49.300763 27513 init.cc:377] Logged in from keytab as kudu/127.24.183.254@KRBTEST.COM (short username kudu)
I20260809 15:12:49.302197 27513 webserver.cc:460] Webserver started at http://127.24.183.254:42887/ using document root <none> and password file <none>
I20260809 15:12:49.302856 27513 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:49.302970 27513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:49.303267 27513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:49.306303 27513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "1644e8e13853479eab701d186d6cab26"
format_stamp: "Formatted at 2026-08-09 15:12:49 on dist-test-slave-nj21"
server_key: "9bdf4f044f5397a9c5564abc6577eea3"
server_key_iv: "f4876a8136dc30650f4157f9fdb01904"
server_key_version: "encryptionkey@0"
I20260809 15:12:49.307046 27513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "1644e8e13853479eab701d186d6cab26"
format_stamp: "Formatted at 2026-08-09 15:12:49 on dist-test-slave-nj21"
server_key: "9bdf4f044f5397a9c5564abc6577eea3"
server_key_iv: "f4876a8136dc30650f4157f9fdb01904"
server_key_version: "encryptionkey@0"
I20260809 15:12:49.312098 27513 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 15:12:49.315446 27541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:49.316713 27513 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:12:49.316874 27513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "1644e8e13853479eab701d186d6cab26"
format_stamp: "Formatted at 2026-08-09 15:12:49 on dist-test-slave-nj21"
server_key: "9bdf4f044f5397a9c5564abc6577eea3"
server_key_iv: "f4876a8136dc30650f4157f9fdb01904"
server_key_version: "encryptionkey@0"
I20260809 15:12:49.317049 27513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:49.328454 27513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:49.332144 27513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:49.332386 27513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:49.349591 27513 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:34781
I20260809 15:12:49.349634 27615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:34781 every 8 connection(s)
I20260809 15:12:49.350948 27513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:12:49.354679 27618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:49.355618 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 27513
I20260809 15:12:49.355813 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:12:49.356261 25311 external_mini_cluster.cc:1428] Setting key b1f5652e6579bd83ef7c60964f5dc489
I20260809 15:12:49.367959 27618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1644e8e13853479eab701d186d6cab26: Bootstrap starting.
I20260809 15:12:49.369385 27622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:49.358024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47139 (local address 127.24.183.254:34781)
0809 15:12:49.358781 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:49.358795 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:49.358836 (+ 41us) server_negotiation.cc:407] Connection header received
0809 15:12:49.360592 (+ 1756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:49.360617 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:49.361008 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:49.361342 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:49.362000 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:49.363210 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:49.365097 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:49.365780 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:49.365900 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:49.367117 (+ 1217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:49.367124 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:49.367136 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:49.367644 (+ 508us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:49.367753 (+ 109us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:49.367784 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:49.367896 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:49.368230 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 15:12:49.368422 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":343,"threads_started":1}
I20260809 15:12:49.371492 27618 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1644e8e13853479eab701d186d6cab26: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:49.372727 27618 log.cc:824] T 00000000000000000000000000000000 P 1644e8e13853479eab701d186d6cab26: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:49.373327 25311 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:12:49.375415 27618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1644e8e13853479eab701d186d6cab26: No bootstrap required, opened a new log
I20260809 15:12:49.384063 27622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:49.377656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54657 (local address 127.24.183.254:34781)
0809 15:12:49.377846 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:49.377852 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:49.378198 (+ 346us) server_negotiation.cc:407] Connection header received
0809 15:12:49.378346 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:49.378351 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:49.378430 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:49.378539 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:49.378965 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:49.380146 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:49.381446 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:49.382140 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:49.382257 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:49.383282 (+ 1025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:49.383288 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:49.383293 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:49.383314 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:49.383365 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:49.383385 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:49.383493 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:49.383635 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 15:12:49.383738 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 15:12:49.384706 27618 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1644e8e13853479eab701d186d6cab26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1644e8e13853479eab701d186d6cab26" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 34781 } }
I20260809 15:12:49.385190 27618 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1644e8e13853479eab701d186d6cab26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:49.385318 27618 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1644e8e13853479eab701d186d6cab26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1644e8e13853479eab701d186d6cab26, State: Initialized, Role: FOLLOWER
I20260809 15:12:49.385890 27618 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1644e8e13853479eab701d186d6cab26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1644e8e13853479eab701d186d6cab26" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 34781 } }
I20260809 15:12:49.386118 27618 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1644e8e13853479eab701d186d6cab26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:12:49.386255 27618 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1644e8e13853479eab701d186d6cab26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:12:49.386409 27618 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1644e8e13853479eab701d186d6cab26 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:49.388825 27618 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1644e8e13853479eab701d186d6cab26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1644e8e13853479eab701d186d6cab26" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 34781 } }
I20260809 15:12:49.389309 27618 leader_election.cc:302] T 00000000000000000000000000000000 P 1644e8e13853479eab701d186d6cab26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1644e8e13853479eab701d186d6cab26; no voters:
I20260809 15:12:49.389894 27618 leader_election.cc:288] T 00000000000000000000000000000000 P 1644e8e13853479eab701d186d6cab26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:12:49.389999 27633 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1644e8e13853479eab701d186d6cab26 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:12:49.390682 27633 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1644e8e13853479eab701d186d6cab26 [term 1 LEADER]: Becoming Leader. State: Replica: 1644e8e13853479eab701d186d6cab26, State: Running, Role: LEADER
I20260809 15:12:49.391309 27633 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1644e8e13853479eab701d186d6cab26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1644e8e13853479eab701d186d6cab26" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 34781 } }
I20260809 15:12:49.391949 27618 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1644e8e13853479eab701d186d6cab26 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:12:49.394094 27634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1644e8e13853479eab701d186d6cab26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1644e8e13853479eab701d186d6cab26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1644e8e13853479eab701d186d6cab26" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 34781 } } }
I20260809 15:12:49.394169 27635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1644e8e13853479eab701d186d6cab26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1644e8e13853479eab701d186d6cab26. Latest consensus state: current_term: 1 leader_uuid: "1644e8e13853479eab701d186d6cab26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1644e8e13853479eab701d186d6cab26" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 34781 } } }
I20260809 15:12:49.394503 27634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1644e8e13853479eab701d186d6cab26 [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:49.394531 27635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1644e8e13853479eab701d186d6cab26 [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:49.396040 27639 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:12:49.400740 27639 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:12:49.408628 27639 catalog_manager.cc:1347] Generated new cluster ID: d0bad89ddbf7443aad57459389f6180f
I20260809 15:12:49.408731 27639 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:12:49.428241 27639 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:12:49.430127 27639 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:12:49.437184 27639 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1644e8e13853479eab701d186d6cab26: Generated new TSK 0
I20260809 15:12:49.437975 27639 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:12:49.446401 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 27513
W20260809 15:12:49.447558 27609 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:47139
W20260809 15:12:49.448356 25311 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 27513: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:12:49.448515 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 27513
W20260809 15:12:49.476186 27495 connection.cc:535] client connection to 127.24.183.254:34781 recv error: Network error: failed to read from TLS socket (remote: 127.24.183.254:34781): Connection reset by peer (error 104)
2026-08-09T15:12:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2067 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:12:49 dist-test-slave-nj21 krb5kdc[27667](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:12:49 dist-test-slave-nj21 krb5kdc[27667](info): set up 2 sockets
Aug 09 15:12:49 dist-test-slave-nj21 krb5kdc[27667](info): commencing operation
krb5kdc: starting...
W20260809 15:12:50.684477 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:12:50 dist-test-slave-nj21 krb5kdc[27667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:12:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:12:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
I20260809 15:12:50.985019 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:35795
--webserver_interface=127.24.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:33995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:35795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:12:51.327140 27685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:51.327414 27685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:51.327517 27685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:51.334702 27685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:12:51.334810 27685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:51.334862 27685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:51.334929 27685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:12:51.334971 27685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:12:51.344993 27685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:33995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:35795
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:35795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:51.346203 27685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:51.347466 27685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:51.358798 27690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:51.358750 27691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:51.358829 27696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:51.359515 27685 server_base.cc:1034] running on GCE node
I20260809 15:12:51.360298 27685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:51.361608 27685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:51.362828 27685 hybrid_clock.cc:648] HybridClock initialized: now 1786288371362788 us; error 31 us; skew 500 ppm
I20260809 15:12:51.363170 27685 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:12:51 dist-test-slave-nj21 krb5kdc[27667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288371, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:12:51.370695 27685 init.cc:377] Logged in from keytab as kudu/127.24.183.254@KRBTEST.COM (short username kudu)
I20260809 15:12:51.372109 27685 webserver.cc:460] Webserver started at http://127.24.183.254:41831/ using document root <none> and password file <none>
I20260809 15:12:51.372727 27685 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:51.372821 27685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:51.373080 27685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:51.375976 27685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "d1906c5a464a4ff38bd0490752323f04"
format_stamp: "Formatted at 2026-08-09 15:12:51 on dist-test-slave-nj21"
server_key: "2d8e121636a82d6cb82a76304a52d1b4"
server_key_iv: "5b06b7f1c1c5d31fe91ecf0113a5a1e4"
server_key_version: "encryptionkey@0"
I20260809 15:12:51.376679 27685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "d1906c5a464a4ff38bd0490752323f04"
format_stamp: "Formatted at 2026-08-09 15:12:51 on dist-test-slave-nj21"
server_key: "2d8e121636a82d6cb82a76304a52d1b4"
server_key_iv: "5b06b7f1c1c5d31fe91ecf0113a5a1e4"
server_key_version: "encryptionkey@0"
I20260809 15:12:51.381529 27685 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 15:12:51.384750 27703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:51.385954 27685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:12:51.386096 27685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "d1906c5a464a4ff38bd0490752323f04"
format_stamp: "Formatted at 2026-08-09 15:12:51 on dist-test-slave-nj21"
server_key: "2d8e121636a82d6cb82a76304a52d1b4"
server_key_iv: "5b06b7f1c1c5d31fe91ecf0113a5a1e4"
server_key_version: "encryptionkey@0"
I20260809 15:12:51.386269 27685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:51.391804 27685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:51.395911 27685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:51.396190 27685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:51.413367 27685 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:35795
I20260809 15:12:51.413384 27784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:35795 every 8 connection(s)
I20260809 15:12:51.414768 27685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:12:51.418303 27785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:51.422760 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 27685
I20260809 15:12:51.422922 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:12:51.423342 25311 external_mini_cluster.cc:1428] Setting key 07a4383c1c82074692005c1a6078fb9e
I20260809 15:12:51.428987 27785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1906c5a464a4ff38bd0490752323f04: Bootstrap starting.
I20260809 15:12:51.430645 27788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:51.425048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45819 (local address 127.24.183.254:35795)
0809 15:12:51.425771 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:51.425785 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:51.425828 (+ 43us) server_negotiation.cc:407] Connection header received
0809 15:12:51.427540 (+ 1712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:51.427562 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:51.427890 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:51.428186 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:51.429191 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:51.429198 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:51.429210 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:51.429224 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:51.429284 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:51.429303 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:51.429499 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:51.429595 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 15:12:51.429756 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":307,"threads_started":1}
I20260809 15:12:51.432260 27785 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1906c5a464a4ff38bd0490752323f04: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:51.433463 27785 log.cc:824] T 00000000000000000000000000000000 P d1906c5a464a4ff38bd0490752323f04: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:51.434285 25311 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:12:51.436062 27785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1906c5a464a4ff38bd0490752323f04: No bootstrap required, opened a new log
I20260809 15:12:51.440867 27788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:51.438522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56489 (local address 127.24.183.254:35795)
0809 15:12:51.438673 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:51.438679 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:51.439184 (+ 505us) server_negotiation.cc:407] Connection header received
0809 15:12:51.439324 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:51.439327 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:51.439377 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:51.439449 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:51.440281 (+ 832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:51.440286 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:51.440288 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:51.440302 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:51.440343 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:51.440359 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:51.440454 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:51.440556 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 15:12:51.440602 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 15:12:51.444209 27785 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1906c5a464a4ff38bd0490752323f04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1906c5a464a4ff38bd0490752323f04" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 35795 } }
I20260809 15:12:51.444573 27785 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1906c5a464a4ff38bd0490752323f04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:51.444666 27785 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1906c5a464a4ff38bd0490752323f04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1906c5a464a4ff38bd0490752323f04, State: Initialized, Role: FOLLOWER
I20260809 15:12:51.445178 27785 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1906c5a464a4ff38bd0490752323f04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1906c5a464a4ff38bd0490752323f04" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 35795 } }
I20260809 15:12:51.445397 27785 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1906c5a464a4ff38bd0490752323f04 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:12:51.445530 27785 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1906c5a464a4ff38bd0490752323f04 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:12:51.445693 27785 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1906c5a464a4ff38bd0490752323f04 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:51.447911 27785 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1906c5a464a4ff38bd0490752323f04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1906c5a464a4ff38bd0490752323f04" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 35795 } }
I20260809 15:12:51.448354 27785 leader_election.cc:302] T 00000000000000000000000000000000 P d1906c5a464a4ff38bd0490752323f04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1906c5a464a4ff38bd0490752323f04; no voters:
I20260809 15:12:51.448947 27785 leader_election.cc:288] T 00000000000000000000000000000000 P d1906c5a464a4ff38bd0490752323f04 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:12:51.449033 27795 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1906c5a464a4ff38bd0490752323f04 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:12:51.449738 27795 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1906c5a464a4ff38bd0490752323f04 [term 1 LEADER]: Becoming Leader. State: Replica: d1906c5a464a4ff38bd0490752323f04, State: Running, Role: LEADER
I20260809 15:12:51.450297 27795 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1906c5a464a4ff38bd0490752323f04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1906c5a464a4ff38bd0490752323f04" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 35795 } }
I20260809 15:12:51.450947 27785 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1906c5a464a4ff38bd0490752323f04 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:12:51.452991 27797 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1906c5a464a4ff38bd0490752323f04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1906c5a464a4ff38bd0490752323f04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1906c5a464a4ff38bd0490752323f04" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 35795 } } }
I20260809 15:12:51.453430 27797 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1906c5a464a4ff38bd0490752323f04 [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:51.453842 27798 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1906c5a464a4ff38bd0490752323f04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1906c5a464a4ff38bd0490752323f04. Latest consensus state: current_term: 1 leader_uuid: "d1906c5a464a4ff38bd0490752323f04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1906c5a464a4ff38bd0490752323f04" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 35795 } } }
I20260809 15:12:51.454198 27798 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1906c5a464a4ff38bd0490752323f04 [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:51.456311 27807 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:12:51.460340 27807 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:12:51.468055 27807 catalog_manager.cc:1347] Generated new cluster ID: 0c5e791afbee45b496b4ffa0df844473
I20260809 15:12:51.468153 27807 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:12:51.479167 27807 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:12:51.479542 27807 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:12:51.487872 27807 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1906c5a464a4ff38bd0490752323f04: Generated new TSK 0
I20260809 15:12:51.488763 27807 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:12:51.493306 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 27685
W20260809 15:12:51.494210 27774 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45819
W20260809 15:12:51.494901 25311 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 27685: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:12:51.495051 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 27685
2026-08-09T15:12:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2046 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:12:51 dist-test-slave-nj21 krb5kdc[27830](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:12:51 dist-test-slave-nj21 krb5kdc[27830](info): set up 2 sockets
Aug 09 15:12:51 dist-test-slave-nj21 krb5kdc[27830](info): commencing operation
krb5kdc: starting...
W20260809 15:12:52.733459 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.005s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:12:52 dist-test-slave-nj21 krb5kdc[27830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:12:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:12:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
I20260809 15:12:53.035347 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:35859
--webserver_interface=127.24.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:34915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:35859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:12:53.378113 27852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:53.378400 27852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:53.378504 27852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:53.385629 27852 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:12:53.385735 27852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:53.385789 27852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:53.385850 27852 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:12:53.385893 27852 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:12:53.395453 27852 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:34915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:35859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:35859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:53.396740 27852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:53.398007 27852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:53.409416 27860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:53.409369 27863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:53.409458 27865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:53.410326 27852 server_base.cc:1034] running on GCE node
I20260809 15:12:53.411114 27852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:53.412498 27852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:53.413697 27852 hybrid_clock.cc:648] HybridClock initialized: now 1786288373413652 us; error 38 us; skew 500 ppm
I20260809 15:12:53.414023 27852 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:12:53 dist-test-slave-nj21 krb5kdc[27830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288373, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:12:53.416992 27852 init.cc:377] Logged in from keytab as kudu/127.24.183.254@KRBTEST.COM (short username kudu)
I20260809 15:12:53.418419 27852 webserver.cc:460] Webserver started at http://127.24.183.254:40927/ using document root <none> and password file <none>
I20260809 15:12:53.419062 27852 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:53.419185 27852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:53.419474 27852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:53.422452 27852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "1175c0f1d6454fd8a7f52988503d56a9"
format_stamp: "Formatted at 2026-08-09 15:12:53 on dist-test-slave-nj21"
server_key: "9e60bc712e042e9696764ea396fe34e5"
server_key_iv: "edc03f9812f8e32dc3e54dde24c0c521"
server_key_version: "encryptionkey@0"
I20260809 15:12:53.423224 27852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "1175c0f1d6454fd8a7f52988503d56a9"
format_stamp: "Formatted at 2026-08-09 15:12:53 on dist-test-slave-nj21"
server_key: "9e60bc712e042e9696764ea396fe34e5"
server_key_iv: "edc03f9812f8e32dc3e54dde24c0c521"
server_key_version: "encryptionkey@0"
I20260809 15:12:53.428153 27852 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260809 15:12:53.431388 27876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:53.432628 27852 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:12:53.432790 27852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "1175c0f1d6454fd8a7f52988503d56a9"
format_stamp: "Formatted at 2026-08-09 15:12:53 on dist-test-slave-nj21"
server_key: "9e60bc712e042e9696764ea396fe34e5"
server_key_iv: "edc03f9812f8e32dc3e54dde24c0c521"
server_key_version: "encryptionkey@0"
I20260809 15:12:53.432981 27852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:53.450829 27852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:53.454491 27852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:53.454746 27852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:53.472126 27852 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:35859
I20260809 15:12:53.472154 27954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:35859 every 8 connection(s)
I20260809 15:12:53.473398 27852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:12:53.473905 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 27852
I20260809 15:12:53.474067 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:12:53.474648 25311 external_mini_cluster.cc:1428] Setting key b44a965b042e04bcbc5c6489bcd41ecf
I20260809 15:12:53.477607 27955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 15:12:53 dist-test-slave-nj21 krb5kdc[27830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:12:53.488031 27955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1175c0f1d6454fd8a7f52988503d56a9: Bootstrap starting.
I20260809 15:12:53.491308 27955 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1175c0f1d6454fd8a7f52988503d56a9: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:53.492548 27955 log.cc:824] T 00000000000000000000000000000000 P 1175c0f1d6454fd8a7f52988503d56a9: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:53.495580 27955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1175c0f1d6454fd8a7f52988503d56a9: No bootstrap required, opened a new log
I20260809 15:12:53.495821 27959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:53.476407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59477 (local address 127.24.183.254:35859)
0809 15:12:53.477164 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:53.477175 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:53.477214 (+ 39us) server_negotiation.cc:407] Connection header received
0809 15:12:53.478883 (+ 1669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:53.478912 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:53.479259 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:53.479577 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:53.481053 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:53.482376 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:53.483611 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:53.484281 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:53.484445 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:53.488767 (+ 4322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:53.488774 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:53.488788 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:12:53.488822 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:53.491746 (+ 2924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:53.492520 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:53.492527 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:53.492532 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:53.492718 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:53.493146 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:53.493151 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:53.493153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:53.493855 (+ 702us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:53.494083 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:53.494506 (+ 423us) server_negotiation.cc:299] Negotiation successful
0809 15:12:53.494798 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":364,"threads_started":1}
I20260809 15:12:53.499471 25311 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:12:53.505129 27955 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1175c0f1d6454fd8a7f52988503d56a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1175c0f1d6454fd8a7f52988503d56a9" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 35859 } }
I20260809 15:12:53.505566 27955 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1175c0f1d6454fd8a7f52988503d56a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:53.505661 27955 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1175c0f1d6454fd8a7f52988503d56a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1175c0f1d6454fd8a7f52988503d56a9, State: Initialized, Role: FOLLOWER
I20260809 15:12:53.506206 27955 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1175c0f1d6454fd8a7f52988503d56a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1175c0f1d6454fd8a7f52988503d56a9" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 35859 } }
I20260809 15:12:53.506434 27955 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1175c0f1d6454fd8a7f52988503d56a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:12:53.506574 27955 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1175c0f1d6454fd8a7f52988503d56a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:12:53.506729 27955 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1175c0f1d6454fd8a7f52988503d56a9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:53.509347 27955 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1175c0f1d6454fd8a7f52988503d56a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1175c0f1d6454fd8a7f52988503d56a9" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 35859 } }
I20260809 15:12:53.509935 27955 leader_election.cc:302] T 00000000000000000000000000000000 P 1175c0f1d6454fd8a7f52988503d56a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1175c0f1d6454fd8a7f52988503d56a9; no voters:
I20260809 15:12:53.510545 27955 leader_election.cc:288] T 00000000000000000000000000000000 P 1175c0f1d6454fd8a7f52988503d56a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:12:53.510643 27969 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1175c0f1d6454fd8a7f52988503d56a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:12:53.511950 27969 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1175c0f1d6454fd8a7f52988503d56a9 [term 1 LEADER]: Becoming Leader. State: Replica: 1175c0f1d6454fd8a7f52988503d56a9, State: Running, Role: LEADER
I20260809 15:12:53.512612 27955 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1175c0f1d6454fd8a7f52988503d56a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:12:53.512570 27969 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1175c0f1d6454fd8a7f52988503d56a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1175c0f1d6454fd8a7f52988503d56a9" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 35859 } }
I20260809 15:12:53.517369 27970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1175c0f1d6454fd8a7f52988503d56a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1175c0f1d6454fd8a7f52988503d56a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1175c0f1d6454fd8a7f52988503d56a9" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 35859 } } }
I20260809 15:12:53.517743 27970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1175c0f1d6454fd8a7f52988503d56a9 [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:53.518293 27971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1175c0f1d6454fd8a7f52988503d56a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1175c0f1d6454fd8a7f52988503d56a9. Latest consensus state: current_term: 1 leader_uuid: "1175c0f1d6454fd8a7f52988503d56a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1175c0f1d6454fd8a7f52988503d56a9" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 35859 } } }
I20260809 15:12:53.518616 27971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1175c0f1d6454fd8a7f52988503d56a9 [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:53.522006 27959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:53.504525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55715 (local address 127.24.183.254:35859)
0809 15:12:53.504690 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:53.504695 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:53.505088 (+ 393us) server_negotiation.cc:407] Connection header received
0809 15:12:53.505210 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:53.505215 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:53.505286 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:53.505392 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:53.506849 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:53.507963 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:53.509113 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:53.509771 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:53.509940 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:53.512705 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:53.512711 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:53.512716 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:12:53.512758 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:53.517666 (+ 4908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:53.518717 (+ 1051us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:53.518723 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:53.518727 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:53.518896 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:53.519344 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:53.519349 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:53.519353 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:53.519774 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:53.521170 (+ 1396us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:53.521355 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 15:12:53.521580 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 15:12:53.529929 27980 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:12:53.534858 27980 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:12:53.543540 27980 catalog_manager.cc:1347] Generated new cluster ID: 30e7e59352dc486aa1f42009189ff056
I20260809 15:12:53.543646 27980 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:12:53.552834 27980 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:12:53.554579 27980 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:12:53.573483 27980 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1175c0f1d6454fd8a7f52988503d56a9: Generated new TSK 0
I20260809 15:12:53.574263 27980 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:12:53.582018 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 27852
I20260809 15:12:53.583139 27947 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:12:53.674539 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 27852
2026-08-09T15:12:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2176 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:12:53 dist-test-slave-nj21 krb5kdc[28001](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:12:53 dist-test-slave-nj21 krb5kdc[28001](info): set up 2 sockets
Aug 09 15:12:53 dist-test-slave-nj21 krb5kdc[28001](info): commencing operation
krb5kdc: starting...
W20260809 15:12:54.912056 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:12:55 dist-test-slave-nj21 krb5kdc[28001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:12:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:12:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
I20260809 15:12:55.214823 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:35291
--webserver_interface=127.24.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:37677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:35291
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:12:55.550424 28026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:55.550709 28026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:55.550873 28026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:55.558094 28026 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:12:55.558211 28026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:55.558290 28026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:55.558369 28026 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:12:55.558437 28026 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:12:55.568118 28026 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:37677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:35291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:35291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:55.569464 28026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:55.570756 28026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:55.582073 28039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:55.582109 28034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:55.582134 28036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:55.583101 28026 server_base.cc:1034] running on GCE node
I20260809 15:12:55.583868 28026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:55.585140 28026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:55.586336 28026 hybrid_clock.cc:648] HybridClock initialized: now 1786288375586294 us; error 35 us; skew 500 ppm
I20260809 15:12:55.586654 28026 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:12:55 dist-test-slave-nj21 krb5kdc[28001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288375, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:12:55.589583 28026 init.cc:377] Logged in from keytab as kudu/127.24.183.254@KRBTEST.COM (short username kudu)
I20260809 15:12:55.590991 28026 webserver.cc:460] Webserver started at http://127.24.183.254:46783/ using document root <none> and password file <none>
I20260809 15:12:55.591624 28026 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:55.591768 28026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:55.592053 28026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:55.594895 28026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "c5ddb245f9954fb6a91d0203bbf1d64a"
format_stamp: "Formatted at 2026-08-09 15:12:55 on dist-test-slave-nj21"
server_key: "f1a8a6e65ca3549ddb12e99762d62657"
server_key_iv: "df0e8db6fcff6931d3e8960f84af1d70"
server_key_version: "encryptionkey@0"
I20260809 15:12:55.595619 28026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "c5ddb245f9954fb6a91d0203bbf1d64a"
format_stamp: "Formatted at 2026-08-09 15:12:55 on dist-test-slave-nj21"
server_key: "f1a8a6e65ca3549ddb12e99762d62657"
server_key_iv: "df0e8db6fcff6931d3e8960f84af1d70"
server_key_version: "encryptionkey@0"
I20260809 15:12:55.600438 28026 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 15:12:55.603533 28049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:55.604758 28026 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 15:12:55.604916 28026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "c5ddb245f9954fb6a91d0203bbf1d64a"
format_stamp: "Formatted at 2026-08-09 15:12:55 on dist-test-slave-nj21"
server_key: "f1a8a6e65ca3549ddb12e99762d62657"
server_key_iv: "df0e8db6fcff6931d3e8960f84af1d70"
server_key_version: "encryptionkey@0"
I20260809 15:12:55.605082 28026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:55.623286 28026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:55.626873 28026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:55.627122 28026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:55.644876 28026 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:35291
I20260809 15:12:55.644894 28122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:35291 every 8 connection(s)
I20260809 15:12:55.646217 28026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:12:55.649900 28124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:55.651695 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 28026
I20260809 15:12:55.651865 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:12:55.652446 25311 external_mini_cluster.cc:1428] Setting key db828ccc76897eb7f138c3bd48fc0c7d
I20260809 15:12:55.663350 28124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5ddb245f9954fb6a91d0203bbf1d64a: Bootstrap starting.
I20260809 15:12:55.665169 28128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:55.654143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46407 (local address 127.24.183.254:35291)
0809 15:12:55.654909 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:55.654922 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:55.654969 (+ 47us) server_negotiation.cc:407] Connection header received
0809 15:12:55.656600 (+ 1631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:55.656623 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:55.657005 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:55.657321 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:55.657898 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:55.659102 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:55.660999 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:55.661670 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:55.661803 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:55.663005 (+ 1202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:55.663013 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:55.663025 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:55.663545 (+ 520us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:55.663625 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:55.663653 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:55.663766 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:55.663990 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 15:12:55.664218 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":351,"threads_started":1}
I20260809 15:12:55.666561 28124 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5ddb245f9954fb6a91d0203bbf1d64a: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:55.667666 28124 log.cc:824] T 00000000000000000000000000000000 P c5ddb245f9954fb6a91d0203bbf1d64a: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:55.668659 25311 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:12:55.670759 28124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5ddb245f9954fb6a91d0203bbf1d64a: No bootstrap required, opened a new log
I20260809 15:12:55.681290 28124 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5ddb245f9954fb6a91d0203bbf1d64a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5ddb245f9954fb6a91d0203bbf1d64a" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 35291 } }
I20260809 15:12:55.681779 28124 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5ddb245f9954fb6a91d0203bbf1d64a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:55.681849 28128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:55.673805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41395 (local address 127.24.183.254:35291)
0809 15:12:55.673996 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:55.674001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:55.674564 (+ 563us) server_negotiation.cc:407] Connection header received
0809 15:12:55.674705 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:55.674710 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:55.674783 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:55.674892 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:55.675721 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:55.676854 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:55.678832 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:55.679546 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:55.679652 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:55.681006 (+ 1354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:55.681013 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:55.681017 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:55.681037 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:55.681084 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:55.681104 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:55.681166 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:55.681422 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 15:12:55.681511 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 15:12:55.681941 28124 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5ddb245f9954fb6a91d0203bbf1d64a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5ddb245f9954fb6a91d0203bbf1d64a, State: Initialized, Role: FOLLOWER
I20260809 15:12:55.682523 28124 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5ddb245f9954fb6a91d0203bbf1d64a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5ddb245f9954fb6a91d0203bbf1d64a" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 35291 } }
I20260809 15:12:55.682754 28124 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5ddb245f9954fb6a91d0203bbf1d64a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:12:55.682874 28124 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5ddb245f9954fb6a91d0203bbf1d64a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:12:55.683012 28124 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5ddb245f9954fb6a91d0203bbf1d64a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:55.685231 28124 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5ddb245f9954fb6a91d0203bbf1d64a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5ddb245f9954fb6a91d0203bbf1d64a" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 35291 } }
I20260809 15:12:55.685665 28124 leader_election.cc:302] T 00000000000000000000000000000000 P c5ddb245f9954fb6a91d0203bbf1d64a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5ddb245f9954fb6a91d0203bbf1d64a; no voters:
I20260809 15:12:55.686192 28124 leader_election.cc:288] T 00000000000000000000000000000000 P c5ddb245f9954fb6a91d0203bbf1d64a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:12:55.686327 28139 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5ddb245f9954fb6a91d0203bbf1d64a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:12:55.686980 28139 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5ddb245f9954fb6a91d0203bbf1d64a [term 1 LEADER]: Becoming Leader. State: Replica: c5ddb245f9954fb6a91d0203bbf1d64a, State: Running, Role: LEADER
I20260809 15:12:55.687502 28139 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5ddb245f9954fb6a91d0203bbf1d64a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5ddb245f9954fb6a91d0203bbf1d64a" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 35291 } }
I20260809 15:12:55.688190 28124 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5ddb245f9954fb6a91d0203bbf1d64a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:12:55.690034 28141 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5ddb245f9954fb6a91d0203bbf1d64a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5ddb245f9954fb6a91d0203bbf1d64a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5ddb245f9954fb6a91d0203bbf1d64a" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 35291 } } }
I20260809 15:12:55.690202 28143 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5ddb245f9954fb6a91d0203bbf1d64a [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5ddb245f9954fb6a91d0203bbf1d64a. Latest consensus state: current_term: 1 leader_uuid: "c5ddb245f9954fb6a91d0203bbf1d64a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5ddb245f9954fb6a91d0203bbf1d64a" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 35291 } } }
I20260809 15:12:55.690511 28141 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5ddb245f9954fb6a91d0203bbf1d64a [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:55.690577 28143 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5ddb245f9954fb6a91d0203bbf1d64a [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:55.692596 28146 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:12:55.698136 28146 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:12:55.707132 28146 catalog_manager.cc:1347] Generated new cluster ID: d03e9d702bff494480e60f401dcb75ce
I20260809 15:12:55.707229 28146 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:12:55.731139 28146 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:12:55.732415 28146 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:12:55.740767 28146 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5ddb245f9954fb6a91d0203bbf1d64a: Generated new TSK 0
I20260809 15:12:55.741503 28146 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:12:55.752288 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 28026
W20260809 15:12:55.753418 28117 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46407
W20260809 15:12:55.754112 25311 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 28026: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:12:55.754246 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 28026
2026-08-09T15:12:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2079 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:12:55 dist-test-slave-nj21 krb5kdc[28169](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:12:55 dist-test-slave-nj21 krb5kdc[28169](info): set up 2 sockets
Aug 09 15:12:55 dist-test-slave-nj21 krb5kdc[28169](info): commencing operation
krb5kdc: starting...
W20260809 15:12:56.993958 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.003s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:12:57 dist-test-slave-nj21 krb5kdc[28169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:12:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:12:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
I20260809 15:12:57.298558 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:40357
--webserver_interface=127.24.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:39257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:40357
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:12:57.635968 28182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:57.636258 28182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:57.636402 28182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:57.643523 28182 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:12:57.643639 28182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:57.643733 28182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:57.643842 28182 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:12:57.643905 28182 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:12:57.653482 28182 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:39257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:40357
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:40357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:57.654691 28182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:57.656060 28182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:57.667655 28194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:57.667703 28192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:57.668303 28198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:57.668820 28182 server_base.cc:1034] running on GCE node
I20260809 15:12:57.669553 28182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:57.670902 28182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:57.672106 28182 hybrid_clock.cc:648] HybridClock initialized: now 1786288377672048 us; error 46 us; skew 500 ppm
I20260809 15:12:57.672430 28182 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:12:57 dist-test-slave-nj21 krb5kdc[28169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288377, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:12:57.679999 28182 init.cc:377] Logged in from keytab as kudu/127.24.183.254@KRBTEST.COM (short username kudu)
I20260809 15:12:57.682171 28182 webserver.cc:460] Webserver started at http://127.24.183.254:39423/ using document root <none> and password file <none>
I20260809 15:12:57.683027 28182 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:57.683138 28182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:57.683411 28182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:57.686417 28182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "1eec7216004946149cb8009d1ad3eba0"
format_stamp: "Formatted at 2026-08-09 15:12:57 on dist-test-slave-nj21"
server_key: "4162b06bd5e9c08261a802ebcdfb24ed"
server_key_iv: "e5cb6d3d0d0eef1b53e3ab4767cfc5e5"
server_key_version: "encryptionkey@0"
I20260809 15:12:57.687140 28182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "1eec7216004946149cb8009d1ad3eba0"
format_stamp: "Formatted at 2026-08-09 15:12:57 on dist-test-slave-nj21"
server_key: "4162b06bd5e9c08261a802ebcdfb24ed"
server_key_iv: "e5cb6d3d0d0eef1b53e3ab4767cfc5e5"
server_key_version: "encryptionkey@0"
I20260809 15:12:57.692233 28182 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 15:12:57.695636 28208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:57.697022 28182 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:12:57.697165 28182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "1eec7216004946149cb8009d1ad3eba0"
format_stamp: "Formatted at 2026-08-09 15:12:57 on dist-test-slave-nj21"
server_key: "4162b06bd5e9c08261a802ebcdfb24ed"
server_key_iv: "e5cb6d3d0d0eef1b53e3ab4767cfc5e5"
server_key_version: "encryptionkey@0"
I20260809 15:12:57.697353 28182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:57.704921 28182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:57.709007 28182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:57.709266 28182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:57.726979 28182 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:40357
I20260809 15:12:57.727002 28287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:40357 every 8 connection(s)
I20260809 15:12:57.728367 28182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:12:57.732000 28288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:57.736084 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 28182
I20260809 15:12:57.736277 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:12:57.736827 25311 external_mini_cluster.cc:1428] Setting key 6b489a41ffc3eaa84b8228c1e7d10ec7
I20260809 15:12:57.744464 28292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:57.738639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47901 (local address 127.24.183.254:40357)
0809 15:12:57.739435 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:57.739447 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:57.739489 (+ 42us) server_negotiation.cc:407] Connection header received
0809 15:12:57.741151 (+ 1662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:57.741174 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:57.741530 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:57.741860 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:57.742923 (+ 1063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:57.742930 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:57.742943 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:57.742957 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:57.743026 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:57.743045 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:57.743217 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:57.743316 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 15:12:57.743495 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":362,"threads_started":1}
I20260809 15:12:57.745188 28288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1eec7216004946149cb8009d1ad3eba0: Bootstrap starting.
I20260809 15:12:57.747918 25311 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:12:57.748338 28288 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1eec7216004946149cb8009d1ad3eba0: Neither blocks nor log segments found. Creating new log.
I20260809 15:12:57.749313 28288 log.cc:824] T 00000000000000000000000000000000 P 1eec7216004946149cb8009d1ad3eba0: Log is configured to *not* fsync() on all Append() calls
I20260809 15:12:57.752290 28288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1eec7216004946149cb8009d1ad3eba0: No bootstrap required, opened a new log
I20260809 15:12:57.754545 28292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:57.752151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32915 (local address 127.24.183.254:40357)
0809 15:12:57.752300 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:57.752306 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:57.752761 (+ 455us) server_negotiation.cc:407] Connection header received
0809 15:12:57.752920 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:57.752924 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:57.752969 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:57.753050 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:57.753932 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:57.753938 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:57.753943 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:12:57.753960 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:57.754004 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:12:57.754022 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:57.754129 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:57.754217 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 15:12:57.754282 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 15:12:57.760263 28288 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1eec7216004946149cb8009d1ad3eba0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eec7216004946149cb8009d1ad3eba0" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 40357 } }
I20260809 15:12:57.760602 28288 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1eec7216004946149cb8009d1ad3eba0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:12:57.760676 28288 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1eec7216004946149cb8009d1ad3eba0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1eec7216004946149cb8009d1ad3eba0, State: Initialized, Role: FOLLOWER
I20260809 15:12:57.761168 28288 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1eec7216004946149cb8009d1ad3eba0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eec7216004946149cb8009d1ad3eba0" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 40357 } }
I20260809 15:12:57.761435 28288 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1eec7216004946149cb8009d1ad3eba0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:12:57.761587 28288 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1eec7216004946149cb8009d1ad3eba0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:12:57.761754 28288 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1eec7216004946149cb8009d1ad3eba0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:12:57.763952 28288 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1eec7216004946149cb8009d1ad3eba0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eec7216004946149cb8009d1ad3eba0" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 40357 } }
I20260809 15:12:57.764412 28288 leader_election.cc:302] T 00000000000000000000000000000000 P 1eec7216004946149cb8009d1ad3eba0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1eec7216004946149cb8009d1ad3eba0; no voters:
I20260809 15:12:57.764961 28288 leader_election.cc:288] T 00000000000000000000000000000000 P 1eec7216004946149cb8009d1ad3eba0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:12:57.765092 28301 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1eec7216004946149cb8009d1ad3eba0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:12:57.765697 28301 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1eec7216004946149cb8009d1ad3eba0 [term 1 LEADER]: Becoming Leader. State: Replica: 1eec7216004946149cb8009d1ad3eba0, State: Running, Role: LEADER
I20260809 15:12:57.766264 28301 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1eec7216004946149cb8009d1ad3eba0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eec7216004946149cb8009d1ad3eba0" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 40357 } }
I20260809 15:12:57.767055 28288 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1eec7216004946149cb8009d1ad3eba0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:12:57.769203 28303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1eec7216004946149cb8009d1ad3eba0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1eec7216004946149cb8009d1ad3eba0. Latest consensus state: current_term: 1 leader_uuid: "1eec7216004946149cb8009d1ad3eba0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eec7216004946149cb8009d1ad3eba0" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 40357 } } }
I20260809 15:12:57.769364 28302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1eec7216004946149cb8009d1ad3eba0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1eec7216004946149cb8009d1ad3eba0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eec7216004946149cb8009d1ad3eba0" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 40357 } } }
I20260809 15:12:57.769646 28303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1eec7216004946149cb8009d1ad3eba0 [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:57.769676 28302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1eec7216004946149cb8009d1ad3eba0 [sys.catalog]: This master's current role is: LEADER
I20260809 15:12:57.771337 28311 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:12:57.776328 28311 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:12:57.784873 28311 catalog_manager.cc:1347] Generated new cluster ID: 7a5a66debfce47c7b761a497ff76072c
I20260809 15:12:57.784992 28311 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:12:57.803985 28311 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:12:57.804463 28311 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:12:57.814033 28311 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1eec7216004946149cb8009d1ad3eba0: Generated new TSK 0
I20260809 15:12:57.814936 28311 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:12:57.820124 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 28182
W20260809 15:12:57.821174 28283 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:47901
W20260809 15:12:57.821848 25311 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 28182: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:12:57.822005 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 28182
2026-08-09T15:12:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2069 ms)
[----------] 18 tests from AuthTokenIssuingTest (37823 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:12:57 dist-test-slave-nj21 krb5kdc[28329](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:12:57 dist-test-slave-nj21 krb5kdc[28329](info): set up 2 sockets
Aug 09 15:12:57 dist-test-slave-nj21 krb5kdc[28329](info): commencing operation
krb5kdc: starting...
W20260809 15:12:59.057436 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.010s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:12:59 dist-test-slave-nj21 krb5kdc[28329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:12:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:12:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
I20260809 15:12:59.370922 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:36287
--webserver_interface=127.24.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:37797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:36287,127.24.183.253:43359,127.24.183.252:41959
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:12:59.708248 28349 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 15:12:59.708577 28349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:12:59.708671 28349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:12:59.708787 28349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:12:59.716040 28349 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:12:59.716122 28349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:12:59.716192 28349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:12:59.716255 28349 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:12:59.716348 28349 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:12:59.726006 28349 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:37797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:36287,127.24.183.253:43359,127.24.183.252:41959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:36287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:12:59.727244 28349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:12:59.728588 28349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:12:59.739594 28359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:59.739640 28362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:12:59.739661 28360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:12:59.741184 28349 server_base.cc:1034] running on GCE node
I20260809 15:12:59.741897 28349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:12:59.743153 28349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:12:59.744364 28349 hybrid_clock.cc:648] HybridClock initialized: now 1786288379744305 us; error 33 us; skew 500 ppm
I20260809 15:12:59.744671 28349 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:12:59 dist-test-slave-nj21 krb5kdc[28329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288379, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:12:59.747720 28349 init.cc:377] Logged in from keytab as kudu/127.24.183.254@KRBTEST.COM (short username kudu)
I20260809 15:12:59.749188 28349 webserver.cc:460] Webserver started at http://127.24.183.254:33353/ using document root <none> and password file <none>
I20260809 15:12:59.749876 28349 fs_manager.cc:359] Metadata directory not provided
I20260809 15:12:59.749977 28349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:12:59.750288 28349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:12:59.753301 28349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "976bba68678c46e48b5606cb471260b4"
format_stamp: "Formatted at 2026-08-09 15:12:59 on dist-test-slave-nj21"
server_key: "7a5a6f2f5c50a88b71fe56302d9be75a"
server_key_iv: "2bee1fb837c8eee8f4c2610f7ed1e885"
server_key_version: "encryptionkey@0"
I20260809 15:12:59.754084 28349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "976bba68678c46e48b5606cb471260b4"
format_stamp: "Formatted at 2026-08-09 15:12:59 on dist-test-slave-nj21"
server_key: "7a5a6f2f5c50a88b71fe56302d9be75a"
server_key_iv: "2bee1fb837c8eee8f4c2610f7ed1e885"
server_key_version: "encryptionkey@0"
I20260809 15:12:59.759631 28349 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 15:12:59.763226 28370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:12:59.764544 28349 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:12:59.764734 28349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "976bba68678c46e48b5606cb471260b4"
format_stamp: "Formatted at 2026-08-09 15:12:59 on dist-test-slave-nj21"
server_key: "7a5a6f2f5c50a88b71fe56302d9be75a"
server_key_iv: "2bee1fb837c8eee8f4c2610f7ed1e885"
server_key_version: "encryptionkey@0"
I20260809 15:12:59.764950 28349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:12:59.791010 28349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:12:59.795226 28349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:12:59.795487 28349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:12:59.815356 28349 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:36287
I20260809 15:12:59.815373 28448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:36287 every 8 connection(s)
I20260809 15:12:59.816679 28349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:12:59.818397 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 28349
I20260809 15:12:59.818630 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:12:59.819476 25311 external_mini_cluster.cc:1428] Setting key 50704505767a82a15bd47c1a07b1cd70
I20260809 15:12:59.821139 28449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:12:59.829625 28449 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36287 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:12:59 dist-test-slave-nj21 krb5kdc[28329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288379, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.183.253@KRBTEST.COM; defaulting to no policy
I20260809 15:12:59.851729 28452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:59.833142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:36287 (local address 127.0.0.1:45298)
0809 15:12:59.834186 (+ 1044us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:59.834226 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 15:12:59.836387 (+ 2161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:12:59.837148 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:12:59.837154 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:12:59.837841 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:12:59.838176 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:59.838193 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:59.839796 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:59.839800 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:59.840858 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:12:59.840870 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:12:59.841773 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:12:59.841781 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:12:59.841955 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:59.842544 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:12:59.842567 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:12:59.844746 (+ 2179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:12:59.847751 (+ 3005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:12:59.847763 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:12:59.847775 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:12:59.848149 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:12:59.848523 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:12:59.848528 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:12:59.848532 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:12:59.848692 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:12:59.849423 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:12:59.849431 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:12:59.849664 (+ 233us) client_negotiation.cc:769] Sending connection context
0809 15:12:59.849905 (+ 241us) client_negotiation.cc:240] Negotiation successful
0809 15:12:59.850488 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":860,"mutex_wait_us":152,"thread_start_us":659,"threads_started":1}
I20260809 15:12:59.851806 28451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:59.833133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45298 (local address 127.24.183.254:36287)
0809 15:12:59.833882 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0809 15:12:59.833897 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 15:12:59.834347 (+ 450us) server_negotiation.cc:407] Connection header received
0809 15:12:59.836717 (+ 2370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:12:59.836722 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:12:59.836929 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:12:59.837099 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:12:59.838379 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:59.839620 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:59.840985 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:12:59.841538 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:12:59.841701 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:12:59.844959 (+ 3258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:12:59.844967 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:12:59.844977 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:12:59.845006 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:12:59.847620 (+ 2614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:59.848269 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:59.848275 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:59.848278 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:59.848419 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:12:59.848793 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:12:59.848797 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:12:59.848799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:12:59.849285 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:12:59.849491 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0809 15:12:59.850050 (+ 559us) server_negotiation.cc:299] Negotiation successful
0809 15:12:59.850318 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":327,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.253@KRBTEST.COM" created.
I20260809 15:12:59.854948 28449 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43359 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:12:59.855981 28452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:59.855554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:43359 (local address 127.0.0.1:36176)
0809 15:12:59.855750 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:59.855866 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.253:43359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 15:12:59.856212 28372 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.183.253:43359: connect: Connection refused (error 111)
I20260809 15:12:59.857678 28452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:59.857308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:43359 (local address 127.0.0.1:36180)
0809 15:12:59.857468 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:59.857539 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.253:43359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260809 15:12:59.857885 28449 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.253:43359: Network error: Client connection negotiation failed: client connection to 127.24.183.253:43359: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.253.keytab.
Entry for principal kudu/127.24.183.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.253.keytab.
I20260809 15:12:59.886988 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.253:43359
--webserver_interface=127.24.183.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.253.keytab
--principal=kudu/127.24.183.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:37797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:36287,127.24.183.253:43359,127.24.183.252:41959
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:12:59.907243 28449 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43359 } attempt: 1
I20260809 15:12:59.908633 28452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:59.908050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:43359 (local address 127.0.0.1:36182)
0809 15:12:59.908289 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:59.908465 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.253:43359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 15:12:59.909659 28452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:59.909239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:43359 (local address 127.0.0.1:36186)
0809 15:12:59.909391 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:59.909491 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.253:43359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 15:12:59.909896 28449 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.253:43359: Network error: Client connection negotiation failed: client connection to 127.24.183.253:43359: connect: Connection refused (error 111)
I20260809 15:12:59.978180 28449 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43359 } attempt: 2
I20260809 15:12:59.979512 28452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:59.978910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:43359 (local address 127.0.0.1:36190)
0809 15:12:59.979189 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:59.979320 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.253:43359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260809 15:12:59.980626 28452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:12:59.980194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:43359 (local address 127.0.0.1:36192)
0809 15:12:59.980369 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 15:12:59.980459 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.253:43359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 15:12:59.980814 28449 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.253:43359: Network error: Client connection negotiation failed: client connection to 127.24.183.253:43359: connect: Connection refused (error 111)
I20260809 15:13:00.072081 28449 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43359 } attempt: 3
I20260809 15:13:00.073357 28452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.072802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:43359 (local address 127.0.0.1:36206)
0809 15:13:00.073042 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:00.073167 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.253:43359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260809 15:13:00.074564 28452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.073986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:43359 (local address 127.0.0.1:36210)
0809 15:13:00.074280 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:00.074379 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.253:43359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
W20260809 15:13:00.074846 28449 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.253:43359: Network error: Client connection negotiation failed: client connection to 127.24.183.253:43359: connect: Connection refused (error 111)
I20260809 15:13:00.218107 28449 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43359 } attempt: 4
I20260809 15:13:00.219426 28452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.218837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:43359 (local address 127.0.0.1:36218)
0809 15:13:00.219175 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:00.219267 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.253:43359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175}
I20260809 15:13:00.220470 28452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.220102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:43359 (local address 127.0.0.1:36228)
0809 15:13:00.220251 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:00.220344 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.253:43359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 15:13:00.220660 28449 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.253:43359: Network error: Client connection negotiation failed: client connection to 127.24.183.253:43359: connect: Connection refused (error 111)
W20260809 15:13:00.230125 28460 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 15:13:00.230420 28460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:00.230515 28460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:00.230623 28460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:00.237885 28460 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:13:00.237973 28460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:00.238065 28460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:00.238127 28460 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:13:00.238217 28460 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:13:00.247859 28460 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:37797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:36287,127.24.183.253:43359,127.24.183.252:41959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.253.keytab
--principal=kudu/127.24.183.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.253:43359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.183.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:00.249085 28460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:00.250412 28460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:00.261673 28467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:00.261756 28466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:00.261840 28471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:00.262770 28460 server_base.cc:1034] running on GCE node
I20260809 15:13:00.263501 28460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:00.264828 28460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:00.266023 28460 hybrid_clock.cc:648] HybridClock initialized: now 1786288380265961 us; error 43 us; skew 500 ppm
I20260809 15:13:00.266322 28460 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:00 dist-test-slave-nj21 krb5kdc[28329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288380, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:00.269447 28460 init.cc:377] Logged in from keytab as kudu/127.24.183.253@KRBTEST.COM (short username kudu)
I20260809 15:13:00.270803 28460 webserver.cc:460] Webserver started at http://127.24.183.253:41415/ using document root <none> and password file <none>
I20260809 15:13:00.271448 28460 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:00.271533 28460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:00.271842 28460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:00.274762 28460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-1/data/instance:
uuid: "de02d1e0f1154387b1706bf12b304df6"
format_stamp: "Formatted at 2026-08-09 15:13:00 on dist-test-slave-nj21"
server_key: "3706758e306e519225c4da3f16847a1b"
server_key_iv: "f953cd614650e77f2e94ddf873a446e4"
server_key_version: "encryptionkey@0"
I20260809 15:13:00.275619 28460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-1/wal/instance:
uuid: "de02d1e0f1154387b1706bf12b304df6"
format_stamp: "Formatted at 2026-08-09 15:13:00 on dist-test-slave-nj21"
server_key: "3706758e306e519225c4da3f16847a1b"
server_key_iv: "f953cd614650e77f2e94ddf873a446e4"
server_key_version: "encryptionkey@0"
I20260809 15:13:00.280462 28460 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 15:13:00.283692 28484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:00.284934 28460 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:13:00.285085 28460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-1/wal
uuid: "de02d1e0f1154387b1706bf12b304df6"
format_stamp: "Formatted at 2026-08-09 15:13:00 on dist-test-slave-nj21"
server_key: "3706758e306e519225c4da3f16847a1b"
server_key_iv: "f953cd614650e77f2e94ddf873a446e4"
server_key_version: "encryptionkey@0"
I20260809 15:13:00.285274 28460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:00.309653 28460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:00.313428 28460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:00.313702 28460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:00.331547 28460 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.253:43359
I20260809 15:13:00.331568 28570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.253:43359 every 8 connection(s)
I20260809 15:13:00.332911 28460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-1/data/info.pb
I20260809 15:13:00.334865 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 28460
I20260809 15:13:00.335034 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-1/wal/instance
I20260809 15:13:00.335506 25311 external_mini_cluster.cc:1428] Setting key 1d2c5fa41a447bb80feef0153cae5031
I20260809 15:13:00.336787 28571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:00.343537 28571 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36287 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:13:00 dist-test-slave-nj21 krb5kdc[28329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288380, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.253@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:13:00.364601 28451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.346092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45314 (local address 127.24.183.254:36287)
0809 15:13:00.346306 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:00.346320 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:00.347578 (+ 1258us) server_negotiation.cc:407] Connection header received
0809 15:13:00.349492 (+ 1914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:00.349497 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:00.349587 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:00.349709 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:00.351566 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:00.352633 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:00.354129 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:00.354766 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:00.354892 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:00.358404 (+ 3512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:00.358411 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:00.358415 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:00.358457 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:00.361680 (+ 3223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:00.362370 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:00.362376 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:00.362380 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:00.362550 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:00.362907 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:00.362911 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:00.362914 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:00.363246 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:00.363392 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:00.364051 (+ 659us) server_negotiation.cc:299] Negotiation successful
0809 15:13:00.364277 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 15:13:00.365391 28573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.346593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:36287 (local address 127.0.0.1:45314)
0809 15:13:00.347412 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:00.347457 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:00.349205 (+ 1748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:00.349955 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:00.349966 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:00.350681 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:00.351387 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:00.351407 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:00.352790 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:00.352794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:00.353963 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:00.353976 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:00.354949 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:00.354953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:00.355147 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:00.355802 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:00.355826 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:00.358148 (+ 2322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:00.361806 (+ 3658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:00.361815 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:00.361829 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:00.362245 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:00.362646 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:00.362649 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:00.362652 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:00.362808 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:00.363353 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:00.363359 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:00.363658 (+ 299us) client_negotiation.cc:769] Sending connection context
0809 15:13:00.363915 (+ 257us) client_negotiation.cc:240] Negotiation successful
0809 15:13:00.364285 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":343,"threads_started":1}
I20260809 15:13:00.367149 28571 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43359 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.24.183.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.252@KRBTEST.COM" created.
Aug 09 15:13:00 dist-test-slave-nj21 krb5kdc[28329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288380, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.253@KRBTEST.COM for kudu/127.24.183.253@KRBTEST.COM
I20260809 15:13:00.380867 28573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.367963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:43359 (local address 127.0.0.1:36242)
0809 15:13:00.368153 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:00.368170 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:00.368270 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:00.369364 (+ 1094us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:00.369369 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:00.369789 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:00.369925 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:00.369933 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:00.371268 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:00.371273 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:00.372205 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:00.372217 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:00.372987 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:00.372992 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:00.373169 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:00.373674 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:00.373691 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:00.375591 (+ 1900us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:00.378474 (+ 2883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:00.378479 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:00.378483 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:00.378823 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:00.379166 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:00.379171 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:00.379175 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:00.379307 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:00.379911 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:00.379915 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:00.380050 (+ 135us) client_negotiation.cc:769] Sending connection context
0809 15:13:00.380165 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 15:13:00.380483 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 15:13:00.380990 28576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.368176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36242 (local address 127.24.183.253:43359)
0809 15:13:00.368898 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:00.368903 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:00.368932 (+ 29us) server_negotiation.cc:407] Connection header received
0809 15:13:00.368982 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:00.368990 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:00.369224 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:00.369355 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:00.370082 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:00.371137 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:00.372333 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:00.372857 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:00.372964 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:00.375809 (+ 2845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:00.375832 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:00.375844 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:00.375872 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:00.378352 (+ 2480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:00.378938 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:00.378946 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:00.378949 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:00.379067 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:00.379411 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:00.379416 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:00.379419 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:00.379788 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:00.379990 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:00.380318 (+ 328us) server_negotiation.cc:299] Negotiation successful
0809 15:13:00.380613 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":570,"thread_start_us":409,"threads_started":1}
I20260809 15:13:00.383355 28571 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.183.252" port: 41959 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:13:00.384547 28573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.384034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:41959 (local address 127.0.0.1:53990)
0809 15:13:00.384187 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:00.384357 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:41959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 15:13:00.384797 28490 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.183.252:41959: connect: Connection refused (error 111)
I20260809 15:13:00.386294 28573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.385981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:41959 (local address 127.0.0.1:53998)
0809 15:13:00.386116 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:00.386192 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:41959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 15:13:00.386528 28571 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.252:41959: Network error: Client connection negotiation failed: client connection to 127.24.183.252:41959: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.252.keytab.
Entry for principal kudu/127.24.183.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.252.keytab.
I20260809 15:13:00.406936 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.252:41959
--webserver_interface=127.24.183.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.252.keytab
--principal=kudu/127.24.183.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:37797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:36287,127.24.183.253:43359,127.24.183.252:41959
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:13:00.435778 28571 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.252" port: 41959 } attempt: 1
I20260809 15:13:00.436911 28573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.436439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:41959 (local address 127.0.0.1:54006)
0809 15:13:00.436638 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:00.436750 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:41959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 15:13:00.437927 28573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.437600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:41959 (local address 127.0.0.1:54014)
0809 15:13:00.437735 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:00.437815 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:41959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 15:13:00.438117 28571 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.252:41959: Network error: Client connection negotiation failed: client connection to 127.24.183.252:41959: connect: Connection refused (error 111)
I20260809 15:13:00.506319 28571 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.252" port: 41959 } attempt: 2
I20260809 15:13:00.507496 28573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.507038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:41959 (local address 127.0.0.1:54020)
0809 15:13:00.507221 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:00.507331 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:41959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 15:13:00.508490 28573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.508174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:41959 (local address 127.0.0.1:54034)
0809 15:13:00.508295 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:00.508375 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:41959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 15:13:00.508697 28571 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.252:41959: Network error: Client connection negotiation failed: client connection to 127.24.183.252:41959: connect: Connection refused (error 111)
I20260809 15:13:00.519901 28449 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43359 } attempt: 5
Aug 09 15:13:00 dist-test-slave-nj21 krb5kdc[28329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288379, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for kudu/127.24.183.253@KRBTEST.COM
I20260809 15:13:00.533497 28576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.520697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36248 (local address 127.24.183.253:43359)
0809 15:13:00.520935 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:00.520941 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:00.520966 (+ 25us) server_negotiation.cc:407] Connection header received
0809 15:13:00.521095 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:00.521101 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:00.521191 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:00.521314 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:00.522096 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:00.523172 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:00.524396 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:00.525071 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:00.525199 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:00.528372 (+ 3173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:00.528377 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:00.528380 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:00.528414 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:00.530879 (+ 2465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:00.531691 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:00.531697 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:00.531700 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:00.531847 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:00.532269 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:00.532273 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:00.532276 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:00.532572 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:00.532684 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:00.533102 (+ 418us) server_negotiation.cc:299] Negotiation successful
0809 15:13:00.533272 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260809 15:13:00.533771 28452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.520614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:43359 (local address 127.0.0.1:36248)
0809 15:13:00.520800 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:00.520840 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:00.520961 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:00.521337 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:00.521340 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:00.521714 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:00.521930 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:00.521941 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:00.523320 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:00.523325 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:00.524240 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:00.524254 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:00.525200 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:00.525204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:00.525356 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:00.525926 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:00.525945 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:00.528189 (+ 2244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:00.531030 (+ 2841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:00.531036 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:00.531039 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:00.531549 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:00.531954 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:00.531959 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:00.531963 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:00.532142 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:00.532685 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:00.532690 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:00.532862 (+ 172us) client_negotiation.cc:769] Sending connection context
0809 15:13:00.533021 (+ 159us) client_negotiation.cc:240] Negotiation successful
0809 15:13:00.533327 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 15:13:00.534845 28449 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.183.252" port: 41959 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:13:00.536170 28452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.535733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:41959 (local address 127.0.0.1:54048)
0809 15:13:00.535910 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:00.536005 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:41959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 15:13:00.537133 28452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.536759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:41959 (local address 127.0.0.1:54060)
0809 15:13:00.536915 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:00.536999 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:41959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 15:13:00.537333 28449 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.252:41959: Network error: Client connection negotiation failed: client connection to 127.24.183.252:41959: connect: Connection refused (error 111)
I20260809 15:13:00.588609 28449 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.252" port: 41959 } attempt: 1
I20260809 15:13:00.589931 28452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.589331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:41959 (local address 127.0.0.1:54064)
0809 15:13:00.589588 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:00.589714 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:41959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260809 15:13:00.591077 28452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.590675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:41959 (local address 127.0.0.1:54078)
0809 15:13:00.590836 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:00.590930 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:41959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 15:13:00.591321 28449 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.252:41959: Network error: Client connection negotiation failed: client connection to 127.24.183.252:41959: connect: Connection refused (error 111)
I20260809 15:13:00.599879 28571 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.252" port: 41959 } attempt: 3
I20260809 15:13:00.600826 28573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.600412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:41959 (local address 127.0.0.1:54092)
0809 15:13:00.600587 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:00.600671 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:41959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 15:13:00.601976 28573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.601532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:41959 (local address 127.0.0.1:54094)
0809 15:13:00.601778 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:00.601861 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:41959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
W20260809 15:13:00.602188 28571 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.252:41959: Network error: Client connection negotiation failed: client connection to 127.24.183.252:41959: connect: Connection refused (error 111)
I20260809 15:13:00.659559 28449 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.252" port: 41959 } attempt: 2
I20260809 15:13:00.660880 28452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.660287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:41959 (local address 127.0.0.1:54096)
0809 15:13:00.660545 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:00.660665 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:41959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260809 15:13:00.661921 28452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.661518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:41959 (local address 127.0.0.1:54100)
0809 15:13:00.661667 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:00.661773 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:41959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 15:13:00.662120 28449 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.252:41959: Network error: Client connection negotiation failed: client connection to 127.24.183.252:41959: connect: Connection refused (error 111)
I20260809 15:13:00.745401 28571 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.252" port: 41959 } attempt: 4
I20260809 15:13:00.746615 28573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.746118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:41959 (local address 127.0.0.1:54114)
0809 15:13:00.746350 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:00.746446 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:41959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 15:13:00.747540 28573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.747241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:41959 (local address 127.0.0.1:54116)
0809 15:13:00.747367 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:00.747436 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:41959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 15:13:00.747756 28571 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.252:41959: Network error: Client connection negotiation failed: client connection to 127.24.183.252:41959: connect: Connection refused (error 111)
W20260809 15:13:00.751544 28581 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 15:13:00.751878 28581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:00.751966 28581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:00.752050 28581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:00.759328 28581 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:13:00.759415 28581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:00.759485 28581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:00.759536 28581 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:13:00.759595 28581 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:13:00.768353 28449 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.252" port: 41959 } attempt: 3
I20260809 15:13:00.769410 28452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.768974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:41959 (local address 127.0.0.1:54124)
0809 15:13:00.769144 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:00.769229 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:41959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 15:13:00.769799 28581 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:37797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:36287,127.24.183.253:43359,127.24.183.252:41959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.252.keytab
--principal=kudu/127.24.183.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.252:41959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.183.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:00.770464 28452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.770112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:41959 (local address 127.0.0.1:54132)
0809 15:13:00.770287 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:00.770365 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:41959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260809 15:13:00.770656 28449 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.252:41959: Network error: Client connection negotiation failed: client connection to 127.24.183.252:41959: connect: Connection refused (error 111)
I20260809 15:13:00.771216 28581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:00.772544 28581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:00.783779 28591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:00.783834 28595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:00.783870 28593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:00.784653 28581 server_base.cc:1034] running on GCE node
I20260809 15:13:00.785362 28581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:00.786623 28581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:00.787776 28581 hybrid_clock.cc:648] HybridClock initialized: now 1786288380787714 us; error 29 us; skew 500 ppm
I20260809 15:13:00.788050 28581 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:00 dist-test-slave-nj21 krb5kdc[28329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288380, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:00.791010 28581 init.cc:377] Logged in from keytab as kudu/127.24.183.252@KRBTEST.COM (short username kudu)
I20260809 15:13:00.792452 28581 webserver.cc:460] Webserver started at http://127.24.183.252:33217/ using document root <none> and password file <none>
I20260809 15:13:00.793102 28581 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:00.793208 28581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:00.793509 28581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:00.796515 28581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-2/data/instance:
uuid: "77d31cda997948b487234917ebd16de1"
format_stamp: "Formatted at 2026-08-09 15:13:00 on dist-test-slave-nj21"
server_key: "4f513c40b424613642430077d68f0313"
server_key_iv: "a12a82cadb9de60aaff327d0f15e8d41"
server_key_version: "encryptionkey@0"
I20260809 15:13:00.797276 28581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-2/wal/instance:
uuid: "77d31cda997948b487234917ebd16de1"
format_stamp: "Formatted at 2026-08-09 15:13:00 on dist-test-slave-nj21"
server_key: "4f513c40b424613642430077d68f0313"
server_key_iv: "a12a82cadb9de60aaff327d0f15e8d41"
server_key_version: "encryptionkey@0"
I20260809 15:13:00.802027 28581 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 15:13:00.805213 28603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:00.806396 28581 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 15:13:00.806547 28581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-2/wal
uuid: "77d31cda997948b487234917ebd16de1"
format_stamp: "Formatted at 2026-08-09 15:13:00 on dist-test-slave-nj21"
server_key: "4f513c40b424613642430077d68f0313"
server_key_iv: "a12a82cadb9de60aaff327d0f15e8d41"
server_key_version: "encryptionkey@0"
I20260809 15:13:00.806743 28581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:00.831557 28581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:00.835361 28581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:00.835636 28581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:00.853013 28581 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.252:41959
I20260809 15:13:00.853034 28674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.252:41959 every 8 connection(s)
I20260809 15:13:00.854274 28581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-2/data/info.pb
I20260809 15:13:00.856101 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 28581
I20260809 15:13:00.856293 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288021313853-25311-0/minicluster-data/master-2/wal/instance
I20260809 15:13:00.856873 25311 external_mini_cluster.cc:1428] Setting key 657b166a9e0e4b1c68692a5dfca52939
I20260809 15:13:00.858573 28679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:00.867110 28679 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36287 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:13:00 dist-test-slave-nj21 krb5kdc[28329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.252@KRBTEST.COM
Aug 09 15:13:00 dist-test-slave-nj21 krb5kdc[28329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288380, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.252@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:13:00.879010 28685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.858713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54146 (local address 127.24.183.252:41959)
0809 15:13:00.859449 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:00.859460 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:00.859501 (+ 41us) server_negotiation.cc:407] Connection header received
0809 15:13:00.861117 (+ 1616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:00.861139 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:00.861494 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:00.861803 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:00.863613 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:00.864983 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:00.866693 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:00.867349 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:00.867471 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:00.872132 (+ 4661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:00.872142 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:00.872155 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:00.872197 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:00.875003 (+ 2806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:00.875619 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:00.875626 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:00.875632 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:00.875835 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:00.876208 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:00.876217 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:00.876220 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:00.876813 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:00.877150 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:00.877465 (+ 315us) server_negotiation.cc:299] Negotiation successful
0809 15:13:00.877877 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":321,"threads_started":1}
I20260809 15:13:00.884069 28687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.869841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:36287 (local address 127.0.0.1:45322)
0809 15:13:00.870666 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:00.870699 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:00.870860 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:00.871217 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:00.871222 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:00.871881 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:00.872171 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:00.872181 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:00.873263 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:00.873267 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:00.874364 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:00.874376 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:00.875107 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:00.875113 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:00.875258 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:00.875791 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:00.875807 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:00.878239 (+ 2432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:00.881301 (+ 3062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:00.881310 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:00.881316 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:00.881726 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:00.882385 (+ 659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:00.882390 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:00.882393 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:00.882565 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:00.883220 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:00.883226 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:00.883380 (+ 154us) client_negotiation.cc:769] Sending connection context
0809 15:13:00.883523 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 15:13:00.883777 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"spinlock_wait_cycles":11136,"thread_start_us":466,"threads_started":1}
I20260809 15:13:00.884354 28686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.869775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45322 (local address 127.24.183.254:36287)
0809 15:13:00.870399 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:00.870405 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:00.870792 (+ 387us) server_negotiation.cc:407] Connection header received
0809 15:13:00.870983 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:00.870987 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:00.871053 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:00.871162 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:00.872328 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:00.873103 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:00.874502 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:00.874923 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:00.875043 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:00.878422 (+ 3379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:00.878427 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:00.878430 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:00.878463 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:00.881152 (+ 2689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:00.882112 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:00.882115 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:00.882118 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:00.882277 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:00.882680 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:00.882684 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:00.882687 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:00.883083 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:00.883234 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:00.883664 (+ 430us) server_negotiation.cc:299] Negotiation successful
0809 15:13:00.883941 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":309,"threads_started":1}
I20260809 15:13:00.885618 28679 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43359 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:13:00 dist-test-slave-nj21 krb5kdc[28329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
Aug 09 15:13:00 dist-test-slave-nj21 krb5kdc[28329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288380, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.252@KRBTEST.COM for kudu/127.24.183.253@KRBTEST.COM
I20260809 15:13:00.897462 28688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.881827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45336 (local address 127.24.183.254:36287)
0809 15:13:00.882441 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:00.882446 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:00.882468 (+ 22us) server_negotiation.cc:407] Connection header received
0809 15:13:00.882514 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:00.882519 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:00.882585 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:00.882681 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:00.884402 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:00.885462 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:00.886952 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:00.887366 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:00.887473 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:00.891947 (+ 4474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:00.891954 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:00.891959 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:00.891998 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:00.894945 (+ 2947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:00.895526 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:00.895531 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:00.895534 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:00.895707 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:00.896032 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:00.896037 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:00.896041 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:00.896389 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:00.896585 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:00.896896 (+ 311us) server_negotiation.cc:299] Negotiation successful
0809 15:13:00.897088 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":259,"threads_started":1}
I20260809 15:13:00.898818 28687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.886252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:43359 (local address 127.0.0.1:36250)
0809 15:13:00.886407 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:00.886424 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:00.886512 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:00.886855 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:00.886860 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:00.887203 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:00.887340 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:00.887348 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:00.888387 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:00.888392 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:00.889808 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:00.889833 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:00.890501 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:00.890507 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:00.890687 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:00.891185 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:00.891202 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:00.893144 (+ 1942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:00.896431 (+ 3287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:00.896435 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:00.896439 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:00.896919 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:00.897338 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:00.897343 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:00.897346 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:00.897490 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:00.898087 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:00.898090 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:00.898211 (+ 121us) client_negotiation.cc:769] Sending connection context
0809 15:13:00.898350 (+ 139us) client_negotiation.cc:240] Negotiation successful
0809 15:13:00.898577 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 15:13:00.898975 28576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.886429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36250 (local address 127.24.183.253:43359)
0809 15:13:00.886587 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:00.886591 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:00.886607 (+ 16us) server_negotiation.cc:407] Connection header received
0809 15:13:00.886657 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:00.886660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:00.886718 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:00.886790 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:00.887487 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:00.888219 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:00.889956 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:00.890370 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:00.890457 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:00.893335 (+ 2878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:00.893341 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:00.893346 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:00.893380 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:00.896303 (+ 2923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:00.897029 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:00.897035 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:00.897039 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:00.897199 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:00.897628 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:00.897633 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:00.897636 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:00.897918 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:00.898010 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:00.898443 (+ 433us) server_negotiation.cc:299] Negotiation successful
0809 15:13:00.898662 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 15:13:00.900049 28679 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.183.252" port: 41959 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:13:00 dist-test-slave-nj21 krb5kdc[28329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.253@KRBTEST.COM
Aug 09 15:13:00 dist-test-slave-nj21 krb5kdc[28329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288380, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.252@KRBTEST.COM for kudu/127.24.183.252@KRBTEST.COM
I20260809 15:13:00.913154 28576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.899085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36258 (local address 127.24.183.253:43359)
0809 15:13:00.899262 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:00.899267 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:00.899283 (+ 16us) server_negotiation.cc:407] Connection header received
0809 15:13:00.899437 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:00.899442 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:00.899516 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:00.899587 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:00.901038 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:00.901727 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:00.902880 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:00.903275 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:00.903387 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:00.907319 (+ 3932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:00.907323 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:00.907326 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:00.907361 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:00.910471 (+ 3110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:00.911233 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:00.911238 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:00.911242 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:00.911402 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:00.911784 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:00.911788 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:00.911792 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:00.912088 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:00.912334 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:00.912636 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 15:13:00.912817 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":400,"threads_started":1}
I20260809 15:13:00.913180 28687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.900773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:41959 (local address 127.0.0.1:54152)
0809 15:13:00.900956 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:00.900969 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:00.901061 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:00.901325 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:00.901330 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:00.901656 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:00.901778 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:00.901786 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:00.902671 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:00.902676 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:00.903510 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:00.903528 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:00.904409 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:00.904414 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:00.904578 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:00.905061 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:00.905076 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:00.907370 (+ 2294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:00.910522 (+ 3152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:00.910528 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:00.910532 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:00.911041 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:00.911402 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:00.911405 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:00.911407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:00.911565 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:00.912209 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:00.912213 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:00.912385 (+ 172us) client_negotiation.cc:769] Sending connection context
0809 15:13:00.912530 (+ 145us) client_negotiation.cc:240] Negotiation successful
0809 15:13:00.912836 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111}
I20260809 15:13:00.913398 28685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.900739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54152 (local address 127.24.183.252:41959)
0809 15:13:00.900915 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:00.900921 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:00.901036 (+ 115us) server_negotiation.cc:407] Connection header received
0809 15:13:00.901151 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:00.901157 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:00.901214 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:00.901287 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:00.901902 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:00.902557 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:00.903641 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:00.904209 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:00.904317 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:00.907534 (+ 3217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:00.907540 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:00.907545 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:00.907577 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:00.910400 (+ 2823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:00.911140 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:00.911143 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:00.911145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:00.911287 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:00.911720 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:00.911725 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:00.911729 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:00.912084 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:00.912223 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:00.912633 (+ 410us) server_negotiation.cc:299] Negotiation successful
0809 15:13:00.913025 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":143,"server-negotiator.queue_time_us":62}
I20260809 15:13:00.917805 25311 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:13:00.918880 28679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77d31cda997948b487234917ebd16de1: Bootstrap starting.
I20260809 15:13:00.922856 28679 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77d31cda997948b487234917ebd16de1: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:00.924166 28679 log.cc:824] T 00000000000000000000000000000000 P 77d31cda997948b487234917ebd16de1: Log is configured to *not* fsync() on all Append() calls
I20260809 15:13:00.921227 28413 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "976bba68678c46e48b5606cb471260b4"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45336
I20260809 15:13:00.927124 28679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77d31cda997948b487234917ebd16de1: No bootstrap required, opened a new log
I20260809 15:13:00.937747 28679 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77d31cda997948b487234917ebd16de1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "976bba68678c46e48b5606cb471260b4" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36287 } } peers { permanent_uuid: "de02d1e0f1154387b1706bf12b304df6" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43359 } } peers { permanent_uuid: "77d31cda997948b487234917ebd16de1" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 41959 } }
I20260809 15:13:00.938118 28679 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77d31cda997948b487234917ebd16de1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77d31cda997948b487234917ebd16de1, State: Initialized, Role: FOLLOWER
I20260809 15:13:00.938604 28679 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77d31cda997948b487234917ebd16de1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "976bba68678c46e48b5606cb471260b4" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36287 } } peers { permanent_uuid: "de02d1e0f1154387b1706bf12b304df6" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43359 } } peers { permanent_uuid: "77d31cda997948b487234917ebd16de1" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 41959 } }
I20260809 15:13:00.939458 28694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77d31cda997948b487234917ebd16de1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "976bba68678c46e48b5606cb471260b4" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36287 } } peers { permanent_uuid: "de02d1e0f1154387b1706bf12b304df6" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43359 } } peers { permanent_uuid: "77d31cda997948b487234917ebd16de1" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 41959 } } }
I20260809 15:13:00.939841 28694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77d31cda997948b487234917ebd16de1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:13:00.940698 28679 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77d31cda997948b487234917ebd16de1 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 15:13:00.949694 28711 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 77d31cda997948b487234917ebd16de1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:13:00.949811 28711 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:13:00.951743 28449 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.252" port: 41959 } attempt: 4
I20260809 15:13:00.947467 28581 master_runner.cc:186] Error getting master registration for 127.24.183.254:36287: OK, instance_id { permanent_uuid: "976bba68678c46e48b5606cb471260b4" instance_seqno: 1786288379808615 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 15:13:00.958798 28581 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 09 15:13:00 dist-test-slave-nj21 krb5kdc[28329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288379, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for kudu/127.24.183.252@KRBTEST.COM
I20260809 15:13:00.968218 28685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.952415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54160 (local address 127.24.183.252:41959)
0809 15:13:00.952602 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:00.952609 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:00.952829 (+ 220us) server_negotiation.cc:407] Connection header received
0809 15:13:00.952887 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:00.952897 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:00.953008 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:00.953102 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:00.953901 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:00.955080 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:00.956948 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:00.957582 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:00.957729 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:00.961630 (+ 3901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:00.961636 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:00.961641 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:00.961679 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:00.965247 (+ 3568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:00.966010 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:00.966016 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:00.966020 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:00.966161 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:00.966567 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:00.966572 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:00.966576 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:00.967018 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:00.967159 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:00.967612 (+ 453us) server_negotiation.cc:299] Negotiation successful
0809 15:13:00.967857 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 15:13:00.968261 28452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:00.952349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:41959 (local address 127.0.0.1:54160)
0809 15:13:00.952589 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:00.952607 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:00.952727 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:00.953149 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:00.953154 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:00.953567 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:00.953745 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:00.953754 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:00.955226 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:00.955233 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:00.956803 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:00.956819 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:00.957764 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:00.957770 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:00.957972 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:00.958591 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:00.958613 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:00.961447 (+ 2834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:00.965386 (+ 3939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:00.965391 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:00.965396 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:00.965904 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:00.966275 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:00.966280 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:00.966284 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:00.966456 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:00.967136 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:00.967142 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:00.967347 (+ 205us) client_negotiation.cc:769] Sending connection context
0809 15:13:00.967513 (+ 166us) client_negotiation.cc:240] Negotiation successful
0809 15:13:00.967837 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133}
I20260809 15:13:00.972973 28449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 976bba68678c46e48b5606cb471260b4: Bootstrap starting.
I20260809 15:13:00.976958 28449 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 976bba68678c46e48b5606cb471260b4: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:00.978222 28449 log.cc:824] T 00000000000000000000000000000000 P 976bba68678c46e48b5606cb471260b4: Log is configured to *not* fsync() on all Append() calls
I20260809 15:13:00.981025 28449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 976bba68678c46e48b5606cb471260b4: No bootstrap required, opened a new log
I20260809 15:13:00.986045 28449 raft_consensus.cc:348] T 00000000000000000000000000000000 P 976bba68678c46e48b5606cb471260b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "976bba68678c46e48b5606cb471260b4" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36287 } } peers { permanent_uuid: "de02d1e0f1154387b1706bf12b304df6" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43359 } } peers { permanent_uuid: "77d31cda997948b487234917ebd16de1" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 41959 } }
I20260809 15:13:00.986368 28449 raft_consensus.cc:729] T 00000000000000000000000000000000 P 976bba68678c46e48b5606cb471260b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 976bba68678c46e48b5606cb471260b4, State: Initialized, Role: FOLLOWER
I20260809 15:13:00.986778 28449 consensus_queue.cc:260] T 00000000000000000000000000000000 P 976bba68678c46e48b5606cb471260b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "976bba68678c46e48b5606cb471260b4" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36287 } } peers { permanent_uuid: "de02d1e0f1154387b1706bf12b304df6" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43359 } } peers { permanent_uuid: "77d31cda997948b487234917ebd16de1" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 41959 } }
I20260809 15:13:00.987664 28715 sys_catalog.cc:455] T 00000000000000000000000000000000 P 976bba68678c46e48b5606cb471260b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "976bba68678c46e48b5606cb471260b4" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36287 } } peers { permanent_uuid: "de02d1e0f1154387b1706bf12b304df6" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43359 } } peers { permanent_uuid: "77d31cda997948b487234917ebd16de1" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 41959 } } }
I20260809 15:13:00.988123 28715 sys_catalog.cc:458] T 00000000000000000000000000000000 P 976bba68678c46e48b5606cb471260b4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:13:00.988927 28449 sys_catalog.cc:564] T 00000000000000000000000000000000 P 976bba68678c46e48b5606cb471260b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:13:00.989089 28413 raft_consensus.cc:482] T 00000000000000000000000000000000 P 976bba68678c46e48b5606cb471260b4 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:13:00.989265 28413 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 976bba68678c46e48b5606cb471260b4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:00.992776 28413 raft_consensus.cc:504] T 00000000000000000000000000000000 P 976bba68678c46e48b5606cb471260b4 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "976bba68678c46e48b5606cb471260b4" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36287 } } peers { permanent_uuid: "de02d1e0f1154387b1706bf12b304df6" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43359 } } peers { permanent_uuid: "77d31cda997948b487234917ebd16de1" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 41959 } }
I20260809 15:13:00.995240 28639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "976bba68678c46e48b5606cb471260b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "77d31cda997948b487234917ebd16de1"
I20260809 15:13:00.995833 28639 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77d31cda997948b487234917ebd16de1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:00.999562 28639 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 77d31cda997948b487234917ebd16de1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 976bba68678c46e48b5606cb471260b4 in term 1.
I20260809 15:13:01.000020 28413 leader_election.cc:288] T 00000000000000000000000000000000 P 976bba68678c46e48b5606cb471260b4 [CANDIDATE]: Term 1 election: Requested vote from peers de02d1e0f1154387b1706bf12b304df6 (127.24.183.253:43359), 77d31cda997948b487234917ebd16de1 (127.24.183.252:41959)
I20260809 15:13:01.001372 28377 leader_election.cc:302] T 00000000000000000000000000000000 P 976bba68678c46e48b5606cb471260b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77d31cda997948b487234917ebd16de1, 976bba68678c46e48b5606cb471260b4; no voters:
I20260809 15:13:01.002002 28715 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 976bba68678c46e48b5606cb471260b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:13:01.013509 28715 raft_consensus.cc:686] T 00000000000000000000000000000000 P 976bba68678c46e48b5606cb471260b4 [term 1 LEADER]: Becoming Leader. State: Replica: 976bba68678c46e48b5606cb471260b4, State: Running, Role: LEADER
I20260809 15:13:01.014580 28715 consensus_queue.cc:237] T 00000000000000000000000000000000 P 976bba68678c46e48b5606cb471260b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "976bba68678c46e48b5606cb471260b4" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36287 } } peers { permanent_uuid: "de02d1e0f1154387b1706bf12b304df6" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43359 } } peers { permanent_uuid: "77d31cda997948b487234917ebd16de1" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 41959 } }
I20260809 15:13:01.028402 28722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 976bba68678c46e48b5606cb471260b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 976bba68678c46e48b5606cb471260b4. Latest consensus state: current_term: 1 leader_uuid: "976bba68678c46e48b5606cb471260b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "976bba68678c46e48b5606cb471260b4" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36287 } } peers { permanent_uuid: "de02d1e0f1154387b1706bf12b304df6" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43359 } } peers { permanent_uuid: "77d31cda997948b487234917ebd16de1" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 41959 } } }
I20260809 15:13:01.028874 28722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 976bba68678c46e48b5606cb471260b4 [sys.catalog]: This master's current role is: LEADER
I20260809 15:13:01.043371 28738 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:13:01.046947 28571 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.252" port: 41959 } attempt: 5
W20260809 15:13:01.048497 28380 tablet.cc:2369] T 00000000000000000000000000000000 P 976bba68678c46e48b5606cb471260b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 15:13:01 dist-test-slave-nj21 krb5kdc[28329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288380, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.253@KRBTEST.COM for kudu/127.24.183.252@KRBTEST.COM
I20260809 15:13:01.059523 28738 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:13:01.069823 28685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:01.047952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54164 (local address 127.24.183.252:41959)
0809 15:13:01.048121 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:01.048126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:01.048146 (+ 20us) server_negotiation.cc:407] Connection header received
0809 15:13:01.049298 (+ 1152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:01.049305 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:01.049401 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:01.049518 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:01.050771 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:01.052384 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:01.054281 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:01.054857 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:01.054975 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:01.061834 (+ 6859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:01.061841 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:01.061845 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:01.061887 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:01.065603 (+ 3716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:01.066554 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:01.066560 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:01.066564 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:01.066742 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:01.067238 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:01.067244 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:01.067247 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:01.067600 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:01.067789 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:01.068322 (+ 533us) server_negotiation.cc:299] Negotiation successful
0809 15:13:01.069300 (+ 978us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 15:13:01.070235 28573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:01.047570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:41959 (local address 127.0.0.1:54164)
0809 15:13:01.047960 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:01.047995 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:01.048121 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:01.049747 (+ 1626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:01.049753 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:01.050338 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:01.050569 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:01.050582 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:01.052569 (+ 1987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:01.052576 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:01.054089 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:01.054106 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:01.055177 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:01.055183 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:01.055427 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:01.056406 (+ 979us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:01.056429 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:01.060821 (+ 4392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:01.065783 (+ 4962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:01.065789 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:01.065794 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:01.066421 (+ 627us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:01.066874 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:01.066880 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:01.066883 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:01.067104 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:01.067854 (+ 750us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:01.067859 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:01.068018 (+ 159us) client_negotiation.cc:769] Sending connection context
0809 15:13:01.068637 (+ 619us) client_negotiation.cc:240] Negotiation successful
0809 15:13:01.068959 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273}
I20260809 15:13:01.081900 28571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de02d1e0f1154387b1706bf12b304df6: Bootstrap starting.
W20260809 15:13:01.091967 28614 tablet.cc:2369] T 00000000000000000000000000000000 P 77d31cda997948b487234917ebd16de1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:13:01.093046 28688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:01.034699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45346 (local address 127.24.183.254:36287)
0809 15:13:01.034904 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:01.034911 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:01.056101 (+ 21190us) server_negotiation.cc:407] Connection header received
0809 15:13:01.056625 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:01.056631 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:01.056716 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:01.056788 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:01.062652 (+ 5864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:01.063745 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:01.068948 (+ 5203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:01.069673 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:01.069775 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:01.079304 (+ 9529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:01.079310 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:01.079315 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:01.079357 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:01.083253 (+ 3896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:01.084600 (+ 1347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:01.084607 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:01.084610 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:01.084772 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:01.085465 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:01.085470 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:01.085474 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:01.085852 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:01.086113 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:01.086974 (+ 861us) server_negotiation.cc:299] Negotiation successful
0809 15:13:01.087198 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 15:13:01.095535 28685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:01.051896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54176 (local address 127.24.183.252:41959)
0809 15:13:01.070654 (+ 18758us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:01.070659 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:01.070676 (+ 17us) server_negotiation.cc:407] Connection header received
0809 15:13:01.070735 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:01.070740 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:01.070811 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:01.070900 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:01.073386 (+ 2486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:01.074454 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:01.083029 (+ 8575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:01.083666 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:01.083796 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:01.086733 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:01.086740 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:01.086744 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:01.086783 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:01.091058 (+ 4275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:01.091903 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:01.091908 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:01.091911 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:01.092058 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:01.092537 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:01.092542 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:01.092545 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:01.092913 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:01.093152 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:01.094877 (+ 1725us) server_negotiation.cc:299] Negotiation successful
0809 15:13:01.095129 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18648,"thread_start_us":24072,"threads_started":1}
I20260809 15:13:01.097020 28576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:01.041579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36262 (local address 127.24.183.253:43359)
0809 15:13:01.041780 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:01.041787 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:01.060040 (+ 18253us) server_negotiation.cc:407] Connection header received
0809 15:13:01.060281 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:01.060287 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:01.060380 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:01.060445 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:01.066443 (+ 5998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:01.067608 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:01.079267 (+ 11659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:01.079955 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:01.080055 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:01.088772 (+ 8717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:01.088779 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:01.088784 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:01.088825 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:01.092556 (+ 3731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:01.093879 (+ 1323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:01.093885 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:01.093887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:01.094075 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:01.094787 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:01.094792 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:01.094795 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:01.095350 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:01.095658 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:01.096263 (+ 605us) server_negotiation.cc:299] Negotiation successful
0809 15:13:01.096618 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 15:13:01.102533 28571 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de02d1e0f1154387b1706bf12b304df6: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:01.104043 28571 log.cc:824] T 00000000000000000000000000000000 P de02d1e0f1154387b1706bf12b304df6: Log is configured to *not* fsync() on all Append() calls
I20260809 15:13:01.087394 28349 master_runner.cc:186] Error getting master registration for 127.24.183.253:43359: OK, instance_id { permanent_uuid: "de02d1e0f1154387b1706bf12b304df6" instance_seqno: 1786288380326688 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 15:13:01.107578 28349 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 15:13:01.109010 28639 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 77d31cda997948b487234917ebd16de1 [term 1 FOLLOWER]: Refusing update from remote peer 976bba68678c46e48b5606cb471260b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:13:01.111255 28722 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 976bba68678c46e48b5606cb471260b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "77d31cda997948b487234917ebd16de1" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 41959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:13:01.116807 28571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de02d1e0f1154387b1706bf12b304df6: No bootstrap required, opened a new log
I20260809 15:13:01.140972 28571 raft_consensus.cc:348] T 00000000000000000000000000000000 P de02d1e0f1154387b1706bf12b304df6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "976bba68678c46e48b5606cb471260b4" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36287 } } peers { permanent_uuid: "de02d1e0f1154387b1706bf12b304df6" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43359 } } peers { permanent_uuid: "77d31cda997948b487234917ebd16de1" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 41959 } }
I20260809 15:13:01.141834 28571 raft_consensus.cc:729] T 00000000000000000000000000000000 P de02d1e0f1154387b1706bf12b304df6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de02d1e0f1154387b1706bf12b304df6, State: Initialized, Role: FOLLOWER
I20260809 15:13:01.142760 28571 consensus_queue.cc:260] T 00000000000000000000000000000000 P de02d1e0f1154387b1706bf12b304df6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "976bba68678c46e48b5606cb471260b4" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36287 } } peers { permanent_uuid: "de02d1e0f1154387b1706bf12b304df6" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43359 } } peers { permanent_uuid: "77d31cda997948b487234917ebd16de1" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 41959 } }
I20260809 15:13:01.143296 28694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77d31cda997948b487234917ebd16de1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 976bba68678c46e48b5606cb471260b4. Latest consensus state: current_term: 1 leader_uuid: "976bba68678c46e48b5606cb471260b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "976bba68678c46e48b5606cb471260b4" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36287 } } peers { permanent_uuid: "de02d1e0f1154387b1706bf12b304df6" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43359 } } peers { permanent_uuid: "77d31cda997948b487234917ebd16de1" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 41959 } } }
I20260809 15:13:01.143659 28694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77d31cda997948b487234917ebd16de1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:13:01.147213 28757 mvcc.cc:205] Tried to move back new op lower bound from 7316637209011142656 to 7316637208646483968. Current Snapshot: MvccSnapshot[applied={T|T < 7316637209011142656}]
I20260809 15:13:01.155850 28536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "976bba68678c46e48b5606cb471260b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "de02d1e0f1154387b1706bf12b304df6"
I20260809 15:13:01.156247 28535 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de02d1e0f1154387b1706bf12b304df6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:01.160172 28535 raft_consensus.cc:1261] T 00000000000000000000000000000000 P de02d1e0f1154387b1706bf12b304df6 [term 1 FOLLOWER]: Refusing update from remote peer 976bba68678c46e48b5606cb471260b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:13:01.163023 28694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77d31cda997948b487234917ebd16de1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "976bba68678c46e48b5606cb471260b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "976bba68678c46e48b5606cb471260b4" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36287 } } peers { permanent_uuid: "de02d1e0f1154387b1706bf12b304df6" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43359 } } peers { permanent_uuid: "77d31cda997948b487234917ebd16de1" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 41959 } } }
I20260809 15:13:01.163280 28694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77d31cda997948b487234917ebd16de1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:13:01.163359 28536 raft_consensus.cc:2454] T 00000000000000000000000000000000 P de02d1e0f1154387b1706bf12b304df6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 976bba68678c46e48b5606cb471260b4 in term 1.
I20260809 15:13:01.163699 28722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 976bba68678c46e48b5606cb471260b4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "976bba68678c46e48b5606cb471260b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "976bba68678c46e48b5606cb471260b4" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36287 } } peers { permanent_uuid: "de02d1e0f1154387b1706bf12b304df6" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43359 } } peers { permanent_uuid: "77d31cda997948b487234917ebd16de1" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 41959 } } }
I20260809 15:13:01.163995 28722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 976bba68678c46e48b5606cb471260b4 [sys.catalog]: This master's current role is: LEADER
I20260809 15:13:01.166033 28571 sys_catalog.cc:564] T 00000000000000000000000000000000 P de02d1e0f1154387b1706bf12b304df6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:13:01.166587 28715 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 976bba68678c46e48b5606cb471260b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de02d1e0f1154387b1706bf12b304df6" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:13:01.167737 28761 sys_catalog.cc:455] T 00000000000000000000000000000000 P de02d1e0f1154387b1706bf12b304df6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "976bba68678c46e48b5606cb471260b4" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36287 } } peers { permanent_uuid: "de02d1e0f1154387b1706bf12b304df6" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43359 } } peers { permanent_uuid: "77d31cda997948b487234917ebd16de1" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 41959 } } }
I20260809 15:13:01.168135 28761 sys_catalog.cc:458] T 00000000000000000000000000000000 P de02d1e0f1154387b1706bf12b304df6 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:13:01.173000 28738 catalog_manager.cc:1347] Generated new cluster ID: 158c4ec42ab54018a7fbb103e8c05d57
I20260809 15:13:01.173103 28738 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:13:01.198549 28738 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:13:01.200673 28738 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:13:01.225989 28738 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 976bba68678c46e48b5606cb471260b4: Generated new TSK 0
I20260809 15:13:01.227308 28738 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:13:01.251443 28761 sys_catalog.cc:455] T 00000000000000000000000000000000 P de02d1e0f1154387b1706bf12b304df6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 976bba68678c46e48b5606cb471260b4. Latest consensus state: current_term: 1 leader_uuid: "976bba68678c46e48b5606cb471260b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "976bba68678c46e48b5606cb471260b4" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36287 } } peers { permanent_uuid: "de02d1e0f1154387b1706bf12b304df6" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43359 } } peers { permanent_uuid: "77d31cda997948b487234917ebd16de1" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 41959 } } }
I20260809 15:13:01.251809 28761 sys_catalog.cc:458] T 00000000000000000000000000000000 P de02d1e0f1154387b1706bf12b304df6 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:13:01.252182 28778 sys_catalog.cc:455] T 00000000000000000000000000000000 P de02d1e0f1154387b1706bf12b304df6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "976bba68678c46e48b5606cb471260b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "976bba68678c46e48b5606cb471260b4" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36287 } } peers { permanent_uuid: "de02d1e0f1154387b1706bf12b304df6" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43359 } } peers { permanent_uuid: "77d31cda997948b487234917ebd16de1" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 41959 } } }
I20260809 15:13:01.252518 28778 sys_catalog.cc:458] T 00000000000000000000000000000000 P de02d1e0f1154387b1706bf12b304df6 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:13:01.253814 28722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 976bba68678c46e48b5606cb471260b4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "976bba68678c46e48b5606cb471260b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "976bba68678c46e48b5606cb471260b4" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 36287 } } peers { permanent_uuid: "de02d1e0f1154387b1706bf12b304df6" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43359 } } peers { permanent_uuid: "77d31cda997948b487234917ebd16de1" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 41959 } } }
I20260809 15:13:01.254094 28722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 976bba68678c46e48b5606cb471260b4 [sys.catalog]: This master's current role is: LEADER
I20260809 15:13:01.294704 28792 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260809 15:13:01.341944 28685 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:13:01.328098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54182 (local address 127.24.183.252:41959)
0809 15:13:01.328280 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:01.328286 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:01.339305 (+ 11019us) server_negotiation.cc:407] Connection header received
0809 15:13:01.340245 (+ 940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:01.340252 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:01.340360 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:01.340442 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:01.341689 (+ 1247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54182: BlockingRecv error: recv got EOF from 127.0.0.1:54182 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260809 15:13:01.344400 28688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:13:01.327745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45356 (local address 127.24.183.254:36287)
0809 15:13:01.327951 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:01.327961 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:01.339905 (+ 11944us) server_negotiation.cc:407] Connection header received
0809 15:13:01.342416 (+ 2511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:01.342422 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:01.342509 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:01.342584 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:01.344107 (+ 1523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45356: BlockingRecv error: recv got EOF from 127.0.0.1:45356 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260809 15:13:01.347993 28576 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:13:01.327318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36268 (local address 127.24.183.253:43359)
0809 15:13:01.327552 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:01.327559 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:01.345717 (+ 18158us) server_negotiation.cc:407] Connection header received
0809 15:13:01.346685 (+ 968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:01.346691 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:01.346773 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:01.346842 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:01.347784 (+ 942us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36268: BlockingRecv error: recv got EOF from 127.0.0.1:36268 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260809 15:13:01.371021 28576 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:13:01.368538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36276 (local address 127.24.183.253:43359)
0809 15:13:01.368807 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:01.368814 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:01.369324 (+ 510us) server_negotiation.cc:407] Connection header received
0809 15:13:01.369554 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:01.369560 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:01.369694 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:01.369813 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:01.370748 (+ 935us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36276: BlockingRecv error: recv got EOF from 127.0.0.1:36276 (error 108)
Metrics: {"server-negotiator.queue_time_us":124}
W20260809 15:13:01.377570 28576 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:13:01.375596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36280 (local address 127.24.183.253:43359)
0809 15:13:01.375808 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:01.375814 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:01.376260 (+ 446us) server_negotiation.cc:407] Connection header received
0809 15:13:01.376451 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:01.376456 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:01.376544 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:01.376630 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:01.377378 (+ 748us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36280: BlockingRecv error: recv got EOF from 127.0.0.1:36280 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260809 15:13:01.389680 28576 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:13:01.385461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36296 (local address 127.24.183.253:43359)
0809 15:13:01.385647 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:01.385654 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:01.388222 (+ 2568us) server_negotiation.cc:407] Connection header received
0809 15:13:01.388427 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:01.388433 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:01.388519 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:01.388617 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:01.389433 (+ 816us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36296: BlockingRecv error: recv got EOF from 127.0.0.1:36296 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260809 15:13:01.399425 28576 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:13:01.397292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36302 (local address 127.24.183.253:43359)
0809 15:13:01.397478 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:01.397484 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:01.397978 (+ 494us) server_negotiation.cc:407] Connection header received
0809 15:13:01.398156 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:01.398162 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:01.398251 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:01.398352 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:01.399180 (+ 828us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36302: BlockingRecv error: recv got EOF from 127.0.0.1:36302 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260809 15:13:01.413326 28576 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:13:01.411167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36318 (local address 127.24.183.253:43359)
0809 15:13:01.411353 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:01.411359 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:01.411892 (+ 533us) server_negotiation.cc:407] Connection header received
0809 15:13:01.412075 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:01.412079 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:01.412167 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:01.412264 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:01.413072 (+ 808us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36318: BlockingRecv error: recv got EOF from 127.0.0.1:36318 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260809 15:13:01.438130 28576 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:13:01.435863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36332 (local address 127.24.183.253:43359)
0809 15:13:01.436045 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:01.436052 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:01.436584 (+ 532us) server_negotiation.cc:407] Connection header received
0809 15:13:01.436788 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:01.436794 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:01.436899 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:01.437008 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:01.437890 (+ 882us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36332: BlockingRecv error: recv got EOF from 127.0.0.1:36332 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260809 15:13:01.480094 28576 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:13:01.477823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36346 (local address 127.24.183.253:43359)
0809 15:13:01.478026 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:01.478033 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:01.478106 (+ 73us) server_negotiation.cc:407] Connection header received
0809 15:13:01.478458 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:01.478464 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:01.478575 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:01.478661 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:01.479835 (+ 1174us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36346: BlockingRecv error: recv got EOF from 127.0.0.1:36346 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260809 15:13:01.550204 28576 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:13:01.547977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36358 (local address 127.24.183.253:43359)
0809 15:13:01.548217 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:01.548224 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:01.548652 (+ 428us) server_negotiation.cc:407] Connection header received
0809 15:13:01.548889 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:01.548894 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:01.548971 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:01.549055 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:01.549988 (+ 933us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36358: BlockingRecv error: recv got EOF from 127.0.0.1:36358 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260809 15:13:01.684238 28576 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:13:01.682222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36368 (local address 127.24.183.253:43359)
0809 15:13:01.682465 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:01.682473 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:01.682740 (+ 267us) server_negotiation.cc:407] Connection header received
0809 15:13:01.682960 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:01.682967 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:01.683063 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:01.683161 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:01.683961 (+ 800us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36368: BlockingRecv error: recv got EOF from 127.0.0.1:36368 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260809 15:13:01.945837 28576 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:13:01.943894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36382 (local address 127.24.183.253:43359)
0809 15:13:01.944347 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:01.944353 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:01.944401 (+ 48us) server_negotiation.cc:407] Connection header received
0809 15:13:01.944626 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:01.944632 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:01.944732 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:01.944855 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:01.945592 (+ 737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36382: BlockingRecv error: recv got EOF from 127.0.0.1:36382 (error 108)
Metrics: {"server-negotiator.queue_time_us":310}
I20260809 15:13:01.951869 28711 catalog_manager.cc:1259] Loaded cluster ID: 158c4ec42ab54018a7fbb103e8c05d57
I20260809 15:13:01.951952 28711 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 77d31cda997948b487234917ebd16de1: loading cluster ID for follower catalog manager: success
I20260809 15:13:01.954398 28711 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 77d31cda997948b487234917ebd16de1: acquiring CA information for follower catalog manager: success
I20260809 15:13:01.956158 28711 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 77d31cda997948b487234917ebd16de1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 15:13:02.299252 28792 catalog_manager.cc:1259] Loaded cluster ID: 158c4ec42ab54018a7fbb103e8c05d57
I20260809 15:13:02.299389 28792 catalog_manager.cc:1552] T 00000000000000000000000000000000 P de02d1e0f1154387b1706bf12b304df6: loading cluster ID for follower catalog manager: success
I20260809 15:13:02.302196 28792 catalog_manager.cc:1574] T 00000000000000000000000000000000 P de02d1e0f1154387b1706bf12b304df6: acquiring CA information for follower catalog manager: success
I20260809 15:13:02.304085 28792 catalog_manager.cc:1602] T 00000000000000000000000000000000 P de02d1e0f1154387b1706bf12b304df6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 15:13:02.472435 28891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:02.465402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36392 (local address 127.24.183.253:43359)
0809 15:13:02.466314 (+ 912us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:02.466321 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:02.466343 (+ 22us) server_negotiation.cc:407] Connection header received
0809 15:13:02.466457 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:02.466464 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:02.466618 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:02.466767 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:13:02.467313 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:02.468392 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:02.469889 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:02.470398 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:02.470526 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:02.471134 (+ 608us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:13:02.471728 (+ 594us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:13:02.471876 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:02.471987 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 15:13:02.472077 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":780,"thread_start_us":573,"threads_started":1}
I20260809 15:13:02.484929 28902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:02.478910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54186 (local address 127.24.183.252:41959)
0809 15:13:02.479575 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:02.479583 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:02.479604 (+ 21us) server_negotiation.cc:407] Connection header received
0809 15:13:02.479749 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:02.479755 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:02.479912 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:02.480076 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:13:02.480550 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:02.481290 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:02.482622 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:02.483055 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:02.483179 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:02.483738 (+ 559us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:13:02.484241 (+ 503us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:13:02.484397 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:02.484529 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 15:13:02.484624 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":407,"threads_started":1}
I20260809 15:13:02.510082 28902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:02.491784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54194 (local address 127.24.183.252:41959)
0809 15:13:02.491943 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:02.491949 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:02.502707 (+ 10758us) server_negotiation.cc:407] Connection header received
0809 15:13:02.502930 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:02.502934 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:02.503020 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:02.503080 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:13:02.504657 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:02.505657 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:02.507797 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:02.508400 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:02.508479 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:02.509239 (+ 760us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:13:02.509387 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:13:02.509467 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:02.509697 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 15:13:02.509778 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 15:13:02.510118 28891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:02.491855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36406 (local address 127.24.183.253:43359)
0809 15:13:02.492027 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:02.492036 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:02.502436 (+ 10400us) server_negotiation.cc:407] Connection header received
0809 15:13:02.502911 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:02.502920 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:02.503010 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:02.503080 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:13:02.504270 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:02.505471 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:02.507719 (+ 2248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:02.508361 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:02.508464 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:02.509247 (+ 783us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:13:02.509379 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:13:02.509458 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:02.509697 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 15:13:02.509782 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 15:13:02.512636 28912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:02.491505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45368 (local address 127.24.183.254:36287)
0809 15:13:02.503971 (+ 12466us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:02.503975 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:02.505136 (+ 1161us) server_negotiation.cc:407] Connection header received
0809 15:13:02.505265 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:02.505271 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:02.505507 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:02.505677 (+ 170us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:13:02.506679 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:02.507815 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:02.509870 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:02.510890 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:02.511005 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:02.511687 (+ 682us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:13:02.512085 (+ 398us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:13:02.512194 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:02.512308 (+ 114us) server_negotiation.cc:299] Negotiation successful
0809 15:13:02.512396 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12348,"thread_start_us":12219,"threads_started":1}
I20260809 15:13:02.517835 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 28349
I20260809 15:13:02.519255 28441 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:13:02.632740 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 28349
I20260809 15:13:02.661010 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 28460
I20260809 15:13:02.662078 28564 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:13:02.770820 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 28460
I20260809 15:13:02.795028 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 28581
I20260809 15:13:02.796205 28669 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:13:02.904247 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 28581
2026-08-09T15:13:02Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5084 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:13:03 dist-test-slave-nj21 krb5kdc[28926](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:13:03 dist-test-slave-nj21 krb5kdc[28926](info): set up 2 sockets
Aug 09 15:13:03 dist-test-slave-nj21 krb5kdc[28926](info): commencing operation
krb5kdc: starting...
W20260809 15:13:04.151647 25311 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.004s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/krb5kdc/test-user.keytab.
Aug 09 15:13:04 dist-test-slave-nj21 krb5kdc[28926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:13:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:13:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
Entry for principal kudu/127.24.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab.
I20260809 15:13:04.458765 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:33563
--webserver_interface=127.24.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:45253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:33563,127.24.183.253:43017,127.24.183.252:35287
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:13:04.791640 28942 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 15:13:04.791980 28942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:04.792084 28942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:04.792191 28942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:04.799309 28942 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:13:04.799394 28942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:04.799479 28942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:04.799547 28942 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:13:04.799631 28942 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:13:04.809336 28942 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:45253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:33563,127.24.183.253:43017,127.24.183.252:35287
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.254.keytab
--principal=kudu/127.24.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.254:33563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:04.810570 28942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:04.811903 28942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:04.823386 28950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:04.823386 28954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:04.823421 28948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:04.824434 28942 server_base.cc:1034] running on GCE node
I20260809 15:13:04.825177 28942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:04.826532 28942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:04.827767 28942 hybrid_clock.cc:648] HybridClock initialized: now 1786288384827684 us; error 49 us; skew 500 ppm
I20260809 15:13:04.828078 28942 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:04 dist-test-slave-nj21 krb5kdc[28926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288384, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:04.831050 28942 init.cc:377] Logged in from keytab as kudu/127.24.183.254@KRBTEST.COM (short username kudu)
I20260809 15:13:04.832463 28942 webserver.cc:460] Webserver started at http://127.24.183.254:37699/ using document root <none> and password file <none>
I20260809 15:13:04.833124 28942 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:04.833223 28942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:04.833527 28942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:04.836429 28942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-0/data/instance:
uuid: "5fdf02368961435f8ba51a79b614d954"
format_stamp: "Formatted at 2026-08-09 15:13:04 on dist-test-slave-nj21"
server_key: "6d051085fd8c4727bbb8655ae1db1085"
server_key_iv: "8203bd8f0b9683dba5f14a74e72c3b63"
server_key_version: "encryptionkey@0"
I20260809 15:13:04.837201 28942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-0/wal/instance:
uuid: "5fdf02368961435f8ba51a79b614d954"
format_stamp: "Formatted at 2026-08-09 15:13:04 on dist-test-slave-nj21"
server_key: "6d051085fd8c4727bbb8655ae1db1085"
server_key_iv: "8203bd8f0b9683dba5f14a74e72c3b63"
server_key_version: "encryptionkey@0"
I20260809 15:13:04.842131 28942 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 15:13:04.845460 28968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:04.846697 28942 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 15:13:04.846865 28942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-0/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-0/wal
uuid: "5fdf02368961435f8ba51a79b614d954"
format_stamp: "Formatted at 2026-08-09 15:13:04 on dist-test-slave-nj21"
server_key: "6d051085fd8c4727bbb8655ae1db1085"
server_key_iv: "8203bd8f0b9683dba5f14a74e72c3b63"
server_key_version: "encryptionkey@0"
I20260809 15:13:04.847055 28942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:04.868147 28942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:04.871860 28942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:04.872099 28942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:04.889986 28942 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.254:33563
I20260809 15:13:04.890004 29040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.254:33563 every 8 connection(s)
I20260809 15:13:04.891283 28942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-0/data/info.pb
I20260809 15:13:04.894425 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 28942
I20260809 15:13:04.894593 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-0/wal/instance
I20260809 15:13:04.895040 29041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:04.895042 25311 external_mini_cluster.cc:1428] Setting key 472f3aafd7a66d0d91924f70cbf13aaf
I20260809 15:13:04.901459 29041 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.183.254" port: 33563 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:13:04 dist-test-slave-nj21 krb5kdc[28926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288384, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:13:04.924554 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:04.904463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:33563 (local address 127.0.0.1:46634)
0809 15:13:04.905322 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:04.905378 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:04.907629 (+ 2251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:04.908485 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:04.908491 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:04.909174 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:04.909606 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:04.909625 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:04.911761 (+ 2136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:04.911765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:04.912876 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:04.912888 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:04.914024 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:04.914029 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:04.914148 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:04.914747 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:04.914769 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:04.917020 (+ 2251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:04.920827 (+ 3807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:04.920846 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:04.920855 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:04.921241 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:04.921688 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:04.921691 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:04.921693 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:04.921850 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:04.922730 (+ 880us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:04.922735 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:04.922941 (+ 206us) client_negotiation.cc:769] Sending connection context
0809 15:13:04.923132 (+ 191us) client_negotiation.cc:240] Negotiation successful
0809 15:13:04.923485 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":351,"threads_started":1}
I20260809 15:13:04.924590 29044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:04.904463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46634 (local address 127.24.183.254:33563)
0809 15:13:04.906191 (+ 1728us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:04.906200 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:04.906243 (+ 43us) server_negotiation.cc:407] Connection header received
0809 15:13:04.908011 (+ 1768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:04.908017 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:04.908238 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:04.908409 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:04.909832 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:04.911572 (+ 1740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:04.913025 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:04.913778 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:04.913986 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:04.917256 (+ 3270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:04.917266 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:04.917281 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:04.917317 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:04.920706 (+ 3389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:04.921391 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:04.921400 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:04.921403 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:04.921586 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:04.921958 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:04.921962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:04.921965 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:04.922616 (+ 651us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:04.922836 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:04.923307 (+ 471us) server_negotiation.cc:299] Negotiation successful
0809 15:13:04.923659 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":71,"server-negotiator.queue_time_us":1544,"thread_start_us":1307,"threads_started":1}
I20260809 15:13:04.927359 29041 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43017 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:13:04.928350 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:04.927989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:43017 (local address 127.0.0.1:37856)
0809 15:13:04.928132 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:04.928236 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.253:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 15:13:04.928565 28969 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.183.253:43017: connect: Connection refused (error 111)
I20260809 15:13:04.930050 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:04.929705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:43017 (local address 127.0.0.1:37872)
0809 15:13:04.929846 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:04.929921 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.253:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 15:13:04.930245 29041 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.253:43017: Network error: Client connection negotiation failed: client connection to 127.24.183.253:43017: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.24.183.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.253.keytab.
Entry for principal kudu/127.24.183.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.253.keytab.
I20260809 15:13:04.966753 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.253:43017
--webserver_interface=127.24.183.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.253.keytab
--principal=kudu/127.24.183.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:45253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:33563,127.24.183.253:43017,127.24.183.252:35287
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:13:04.979534 29041 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43017 } attempt: 1
I20260809 15:13:04.980664 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:04.980215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:43017 (local address 127.0.0.1:37882)
0809 15:13:04.980409 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:04.980515 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.253:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 15:13:04.981658 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:04.981299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:43017 (local address 127.0.0.1:37892)
0809 15:13:04.981455 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:04.981523 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.253:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260809 15:13:04.981853 29041 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.253:43017: Network error: Client connection negotiation failed: client connection to 127.24.183.253:43017: connect: Connection refused (error 111)
I20260809 15:13:05.050066 29041 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43017 } attempt: 2
I20260809 15:13:05.051218 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.050721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:43017 (local address 127.0.0.1:37908)
0809 15:13:05.050970 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:05.051062 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.253:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 15:13:05.052956 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.052047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:43017 (local address 127.0.0.1:37924)
0809 15:13:05.052725 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:05.052809 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.253:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":577}
W20260809 15:13:05.053130 29041 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.253:43017: Network error: Client connection negotiation failed: client connection to 127.24.183.253:43017: connect: Connection refused (error 111)
I20260809 15:13:05.144387 29041 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43017 } attempt: 3
I20260809 15:13:05.145527 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.145083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:43017 (local address 127.0.0.1:37932)
0809 15:13:05.145287 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:05.145384 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.253:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260809 15:13:05.146675 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.146285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:43017 (local address 127.0.0.1:37934)
0809 15:13:05.146454 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:05.146526 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.253:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260809 15:13:05.146878 29041 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.253:43017: Network error: Client connection negotiation failed: client connection to 127.24.183.253:43017: connect: Connection refused (error 111)
I20260809 15:13:05.290108 29041 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43017 } attempt: 4
I20260809 15:13:05.291253 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.290729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:43017 (local address 127.0.0.1:37944)
0809 15:13:05.291008 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:05.291101 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.253:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
I20260809 15:13:05.292338 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.291970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:43017 (local address 127.0.0.1:37946)
0809 15:13:05.292116 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:05.292202 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.253:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 15:13:05.292516 29041 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.253:43017: Network error: Client connection negotiation failed: client connection to 127.24.183.253:43017: connect: Connection refused (error 111)
W20260809 15:13:05.296125 29051 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 15:13:05.296437 29051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:05.296535 29051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:05.296653 29051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:05.303815 29051 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:13:05.303898 29051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:05.303987 29051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:05.304051 29051 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:13:05.304134 29051 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:13:05.313714 29051 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:45253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:33563,127.24.183.253:43017,127.24.183.252:35287
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.253.keytab
--principal=kudu/127.24.183.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.253:43017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.183.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:05.314934 29051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:05.316205 29051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:05.327440 29056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:05.327471 29057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:05.327378 29059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:05.327996 29051 server_base.cc:1034] running on GCE node
I20260809 15:13:05.328743 29051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:05.330273 29051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:05.331521 29051 hybrid_clock.cc:648] HybridClock initialized: now 1786288385331438 us; error 48 us; skew 500 ppm
I20260809 15:13:05.331868 29051 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:05 dist-test-slave-nj21 krb5kdc[28926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288385, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:05.334800 29051 init.cc:377] Logged in from keytab as kudu/127.24.183.253@KRBTEST.COM (short username kudu)
I20260809 15:13:05.336112 29051 webserver.cc:460] Webserver started at http://127.24.183.253:43247/ using document root <none> and password file <none>
I20260809 15:13:05.336751 29051 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:05.336849 29051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:05.337147 29051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:05.340044 29051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-1/data/instance:
uuid: "524c2e0bee4f4a5aa7ff39c8727aaff2"
format_stamp: "Formatted at 2026-08-09 15:13:05 on dist-test-slave-nj21"
server_key: "804302a8b2f5a540e416abbf6a402aa8"
server_key_iv: "0811a3bba28902246db8b667adaed810"
server_key_version: "encryptionkey@0"
I20260809 15:13:05.340811 29051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-1/wal/instance:
uuid: "524c2e0bee4f4a5aa7ff39c8727aaff2"
format_stamp: "Formatted at 2026-08-09 15:13:05 on dist-test-slave-nj21"
server_key: "804302a8b2f5a540e416abbf6a402aa8"
server_key_iv: "0811a3bba28902246db8b667adaed810"
server_key_version: "encryptionkey@0"
I20260809 15:13:05.345702 29051 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 15:13:05.348908 29068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:05.350189 29051 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:13:05.350344 29051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-1/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-1/wal
uuid: "524c2e0bee4f4a5aa7ff39c8727aaff2"
format_stamp: "Formatted at 2026-08-09 15:13:05 on dist-test-slave-nj21"
server_key: "804302a8b2f5a540e416abbf6a402aa8"
server_key_iv: "0811a3bba28902246db8b667adaed810"
server_key_version: "encryptionkey@0"
I20260809 15:13:05.350525 29051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:05.371625 29051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:05.375274 29051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:05.375514 29051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:05.392910 29051 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.253:43017
I20260809 15:13:05.392935 29146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.253:43017 every 8 connection(s)
I20260809 15:13:05.394281 29051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-1/data/info.pb
I20260809 15:13:05.397852 29147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:05.403651 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 29051
I20260809 15:13:05.403877 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-1/wal/instance
I20260809 15:13:05.404300 25311 external_mini_cluster.cc:1428] Setting key aa69288298df8f6ace3c8195406a0082
I20260809 15:13:05.404316 29147 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.183.254" port: 33563 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:13:05 dist-test-slave-nj21 krb5kdc[28926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288385, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.253@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:13:05.424001 29044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.406883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46642 (local address 127.24.183.254:33563)
0809 15:13:05.407060 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:05.407064 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:05.408288 (+ 1224us) server_negotiation.cc:407] Connection header received
0809 15:13:05.410153 (+ 1865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:05.410161 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:05.410243 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:05.410333 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:05.411820 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:05.412522 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:05.414008 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:05.414428 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:05.414546 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:05.417850 (+ 3304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:05.417855 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:05.417858 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:05.417894 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:05.421049 (+ 3155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:05.421767 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:05.421773 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:05.421777 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:05.421957 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:05.422333 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:05.422344 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:05.422347 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:05.422679 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:05.422825 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:05.423460 (+ 635us) server_negotiation.cc:299] Negotiation successful
0809 15:13:05.423707 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 15:13:05.424723 29151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.407366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:33563 (local address 127.0.0.1:46642)
0809 15:13:05.408126 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:05.408167 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:05.409880 (+ 1713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:05.410568 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:05.410576 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:05.411091 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:05.411627 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:05.411644 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:05.412676 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:05.412680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:05.413850 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:05.413869 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:05.414652 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:05.414659 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:05.414889 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:05.415495 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:05.415519 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:05.417631 (+ 2112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:05.421180 (+ 3549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:05.421187 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:05.421197 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:05.421624 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:05.422056 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:05.422059 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:05.422061 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:05.422219 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:05.422781 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:05.422787 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:05.423074 (+ 287us) client_negotiation.cc:769] Sending connection context
0809 15:13:05.423288 (+ 214us) client_negotiation.cc:240] Negotiation successful
0809 15:13:05.423635 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":319,"threads_started":1}
I20260809 15:13:05.426410 29147 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43017 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:13:05 dist-test-slave-nj21 krb5kdc[28926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288385, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.253@KRBTEST.COM for kudu/127.24.183.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.183.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.183.252@KRBTEST.COM" created.
I20260809 15:13:05.440937 29151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.427011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:43017 (local address 127.0.0.1:37950)
0809 15:13:05.427167 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:05.427186 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:05.427289 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:05.428229 (+ 940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:05.428234 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:05.428616 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:05.428748 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:05.428756 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:05.430450 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:05.430455 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:05.431663 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:05.431697 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:05.432472 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:05.432482 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:05.432644 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:05.433155 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:05.433178 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:05.435650 (+ 2472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:05.438788 (+ 3138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:05.438792 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:05.438795 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:05.439140 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:05.439478 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:05.439481 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:05.439483 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:05.439611 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:05.440237 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:05.440241 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:05.440362 (+ 121us) client_negotiation.cc:769] Sending connection context
0809 15:13:05.440473 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 15:13:05.440669 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 15:13:05.441329 29153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.427261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37950 (local address 127.24.183.253:43017)
0809 15:13:05.427865 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:05.427870 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:05.427894 (+ 24us) server_negotiation.cc:407] Connection header received
0809 15:13:05.427935 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:05.427940 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:05.428087 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:05.428216 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:05.428900 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:05.430293 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:05.431804 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:05.432332 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:05.432424 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:05.435868 (+ 3444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:05.435879 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:05.435893 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:05.435922 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:05.438667 (+ 2745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:05.439250 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:05.439256 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:05.439258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:05.439394 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:05.439717 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:05.439721 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:05.439724 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:05.440118 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:05.440308 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:05.440653 (+ 345us) server_negotiation.cc:299] Negotiation successful
0809 15:13:05.440890 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":291,"threads_started":1}
I20260809 15:13:05.443706 29147 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.183.252" port: 35287 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:13:05.444684 29151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.444300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:35287 (local address 127.0.0.1:53486)
0809 15:13:05.444446 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:05.444545 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:35287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 15:13:05.444908 29070 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.183.252:35287: connect: Connection refused (error 111)
I20260809 15:13:05.446346 29151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.446024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:35287 (local address 127.0.0.1:53498)
0809 15:13:05.446169 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:05.446246 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:35287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 15:13:05.446545 29147 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.252:35287: Network error: Client connection negotiation failed: client connection to 127.24.183.252:35287: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.183.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.252.keytab.
Entry for principal kudu/127.24.183.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.252.keytab.
I20260809 15:13:05.474835 25311 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
/tmp/dist-test-taskXnZFlc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.252:35287
--webserver_interface=127.24.183.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.252.keytab
--principal=kudu/127.24.183.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.183.212:45253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.183.254:33563,127.24.183.253:43017,127.24.183.252:35287
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:13:05.495839 29147 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.252" port: 35287 } attempt: 1
I20260809 15:13:05.496968 29151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.496490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:35287 (local address 127.0.0.1:53512)
0809 15:13:05.496685 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:05.496804 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:35287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 15:13:05.498095 29151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.497611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:35287 (local address 127.0.0.1:53518)
0809 15:13:05.497846 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:05.497948 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:35287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260809 15:13:05.498338 29147 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.252:35287: Network error: Client connection negotiation failed: client connection to 127.24.183.252:35287: connect: Connection refused (error 111)
I20260809 15:13:05.566560 29147 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.252" port: 35287 } attempt: 2
I20260809 15:13:05.567811 29151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.567196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:35287 (local address 127.0.0.1:53522)
0809 15:13:05.567444 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:05.567567 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:35287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 15:13:05.568940 29151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.568516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:35287 (local address 127.0.0.1:53532)
0809 15:13:05.568679 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:05.568769 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:35287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 15:13:05.569185 29147 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.252:35287: Network error: Client connection negotiation failed: client connection to 127.24.183.252:35287: connect: Connection refused (error 111)
I20260809 15:13:05.591790 29041 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43017 } attempt: 5
Aug 09 15:13:05 dist-test-slave-nj21 krb5kdc[28926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288384, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for kudu/127.24.183.253@KRBTEST.COM
I20260809 15:13:05.606248 29153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.592703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37964 (local address 127.24.183.253:43017)
0809 15:13:05.592875 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:05.592880 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:05.592902 (+ 22us) server_negotiation.cc:407] Connection header received
0809 15:13:05.593024 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:05.593028 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:05.593109 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:05.593191 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:05.594208 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:05.594965 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:05.596648 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:05.597076 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:05.597172 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:05.601077 (+ 3905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:05.601082 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:05.601085 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:05.601121 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:05.603598 (+ 2477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:05.604452 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:05.604456 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:05.604458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:05.604585 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:05.604994 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:05.604996 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:05.604998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:05.605275 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:05.605390 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:05.605837 (+ 447us) server_negotiation.cc:299] Negotiation successful
0809 15:13:05.606008 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 15:13:05.606418 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.592475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:43017 (local address 127.0.0.1:37964)
0809 15:13:05.592721 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:05.592744 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:05.592903 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:05.593254 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:05.593260 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:05.593767 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:05.594038 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:05.594052 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:05.595104 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:05.595110 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:05.596476 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:05.596494 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:05.597233 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:05.597239 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:05.597481 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:05.598087 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:05.598110 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:05.600872 (+ 2762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:05.603770 (+ 2898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:05.603776 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:05.603781 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:05.604337 (+ 556us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:05.604687 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:05.604693 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:05.604697 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:05.604894 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:05.605389 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:05.605395 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:05.605579 (+ 184us) client_negotiation.cc:769] Sending connection context
0809 15:13:05.605744 (+ 165us) client_negotiation.cc:240] Negotiation successful
0809 15:13:05.606076 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260809 15:13:05.607457 29041 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.183.252" port: 35287 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:13:05.608481 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.608136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:35287 (local address 127.0.0.1:53538)
0809 15:13:05.608279 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:05.608353 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:35287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 15:13:05.609382 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.609072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:35287 (local address 127.0.0.1:53546)
0809 15:13:05.609177 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:05.609271 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:35287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 15:13:05.609580 29041 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.252:35287: Network error: Client connection negotiation failed: client connection to 127.24.183.252:35287: connect: Connection refused (error 111)
I20260809 15:13:05.660396 29147 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.252" port: 35287 } attempt: 3
I20260809 15:13:05.660825 29041 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.252" port: 35287 } attempt: 1
I20260809 15:13:05.661598 29151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.661101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:35287 (local address 127.0.0.1:53558)
0809 15:13:05.661305 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:05.661398 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:35287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 15:13:05.662129 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.661427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:35287 (local address 127.0.0.1:53564)
0809 15:13:05.661622 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:05.661946 (+ 324us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:35287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 15:13:05.662794 29151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.662277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:35287 (local address 127.0.0.1:53576)
0809 15:13:05.662587 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:05.662682 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:35287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203}
W20260809 15:13:05.663069 29147 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.252:35287: Network error: Client connection negotiation failed: client connection to 127.24.183.252:35287: connect: Connection refused (error 111)
I20260809 15:13:05.663144 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.662793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:35287 (local address 127.0.0.1:53584)
0809 15:13:05.662947 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:05.663031 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:35287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 15:13:05.663352 29041 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.252:35287: Network error: Client connection negotiation failed: client connection to 127.24.183.252:35287: connect: Connection refused (error 111)
I20260809 15:13:05.731552 29041 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.252" port: 35287 } attempt: 2
I20260809 15:13:05.732818 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.732306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:35287 (local address 127.0.0.1:53600)
0809 15:13:05.732548 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:05.732646 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:35287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 15:13:05.733786 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.733474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:35287 (local address 127.0.0.1:53612)
0809 15:13:05.733590 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:05.733673 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:35287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 15:13:05.733994 29041 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.252:35287: Network error: Client connection negotiation failed: client connection to 127.24.183.252:35287: connect: Connection refused (error 111)
I20260809 15:13:05.806324 29147 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.252" port: 35287 } attempt: 4
I20260809 15:13:05.807516 29151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.807006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:35287 (local address 127.0.0.1:53626)
0809 15:13:05.807260 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:05.807351 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:35287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260809 15:13:05.808579 29151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.808245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:35287 (local address 127.0.0.1:53638)
0809 15:13:05.808364 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:05.808441 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:35287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 15:13:05.808833 29147 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.252:35287: Network error: Client connection negotiation failed: client connection to 127.24.183.252:35287: connect: Connection refused (error 111)
W20260809 15:13:05.816104 29164 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 15:13:05.816407 29164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:05.816506 29164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:05.816617 29164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:05.823773 29164 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:13:05.823858 29164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:05.823952 29164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:05.824018 29164 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:13:05.824103 29164 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:13:05.833918 29164 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.183.212:45253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.183.254:33563,127.24.183.253:43017,127.24.183.252:35287
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/krb5kdc/kudu_127.24.183.252.keytab
--principal=kudu/127.24.183.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.183.252:35287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.183.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:05.835212 29164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:05.836563 29164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 15:13:05.840191 29041 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.252" port: 35287 } attempt: 3
I20260809 15:13:05.841403 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.840885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:35287 (local address 127.0.0.1:53644)
0809 15:13:05.841067 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:05.841162 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:35287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 15:13:05.842801 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.842352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:35287 (local address 127.0.0.1:53652)
0809 15:13:05.842553 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:05.842653 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.183.252:35287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260809 15:13:05.843086 29041 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.183.252:35287: Network error: Client connection negotiation failed: client connection to 127.24.183.252:35287: connect: Connection refused (error 111)
W20260809 15:13:05.848197 29175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:05.848354 29172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:05.848238 29173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:05.849241 29164 server_base.cc:1034] running on GCE node
I20260809 15:13:05.850050 29164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:05.851410 29164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:05.852640 29164 hybrid_clock.cc:648] HybridClock initialized: now 1786288385852575 us; error 36 us; skew 500 ppm
I20260809 15:13:05.852968 29164 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:05 dist-test-slave-nj21 krb5kdc[28926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288385, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:05.856155 29164 init.cc:377] Logged in from keytab as kudu/127.24.183.252@KRBTEST.COM (short username kudu)
I20260809 15:13:05.857507 29164 webserver.cc:460] Webserver started at http://127.24.183.252:37895/ using document root <none> and password file <none>
I20260809 15:13:05.858155 29164 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:05.858245 29164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:05.858559 29164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:05.861480 29164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-2/data/instance:
uuid: "f1a43852e60a4bb5b1210021a219aa88"
format_stamp: "Formatted at 2026-08-09 15:13:05 on dist-test-slave-nj21"
server_key: "7e5b908f76cbc58dab834dd17da08c2e"
server_key_iv: "1ed2a2e3814147b6249cbe4141dbe773"
server_key_version: "encryptionkey@0"
I20260809 15:13:05.862247 29164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-2/wal/instance:
uuid: "f1a43852e60a4bb5b1210021a219aa88"
format_stamp: "Formatted at 2026-08-09 15:13:05 on dist-test-slave-nj21"
server_key: "7e5b908f76cbc58dab834dd17da08c2e"
server_key_iv: "1ed2a2e3814147b6249cbe4141dbe773"
server_key_version: "encryptionkey@0"
I20260809 15:13:05.867147 29164 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 15:13:05.870458 29185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:05.871740 29164 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:13:05.871899 29164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-2/data,/tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-2/wal
uuid: "f1a43852e60a4bb5b1210021a219aa88"
format_stamp: "Formatted at 2026-08-09 15:13:05 on dist-test-slave-nj21"
server_key: "7e5b908f76cbc58dab834dd17da08c2e"
server_key_iv: "1ed2a2e3814147b6249cbe4141dbe773"
server_key_version: "encryptionkey@0"
I20260809 15:13:05.872088 29164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:05.898389 29164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:05.902311 29164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:05.902552 29164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:05.920238 29164 rpc_server.cc:307] RPC server started. Bound to: 127.24.183.252:35287
I20260809 15:13:05.920248 29251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.183.252:35287 every 8 connection(s)
I20260809 15:13:05.921482 29164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-2/data/info.pb
I20260809 15:13:05.924249 25311 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu as pid 29164
I20260809 15:13:05.924455 25311 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXnZFlc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288021313853-25311-0/minicluster-data/master-2/wal/instance
I20260809 15:13:05.925005 25311 external_mini_cluster.cc:1428] Setting key 5471baa55ce1efa781a967fb578aa604
I20260809 15:13:05.925650 29252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:05.935151 29252 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.183.254" port: 33563 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:13:05 dist-test-slave-nj21 krb5kdc[28926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.252@KRBTEST.COM
Aug 09 15:13:05 dist-test-slave-nj21 krb5kdc[28926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288385, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.252@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
I20260809 15:13:05.947535 29255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.926736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53654 (local address 127.24.183.252:35287)
0809 15:13:05.927484 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:05.927496 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:05.927538 (+ 42us) server_negotiation.cc:407] Connection header received
0809 15:13:05.929082 (+ 1544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:05.929104 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:05.929488 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:05.929805 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:05.931553 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:05.932914 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:05.934683 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:05.935331 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:05.935477 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:05.940156 (+ 4679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:05.940165 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:05.940179 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:05.940223 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:05.943362 (+ 3139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:05.944114 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:05.944120 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:05.944126 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:05.944344 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:05.944717 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:05.944721 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:05.944723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:05.945229 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:05.945546 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:05.945947 (+ 401us) server_negotiation.cc:299] Negotiation successful
0809 15:13:05.946338 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"spinlock_wait_cycles":12160,"thread_start_us":320,"threads_started":1}
I20260809 15:13:05.952852 29256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.937975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.254:33563 (local address 127.0.0.1:46652)
0809 15:13:05.938631 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:05.938678 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:05.938812 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:05.939537 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:05.939544 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:05.940254 (+ 710us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:05.940634 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:05.940647 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:05.941708 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:05.941714 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:05.942874 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:05.942887 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:05.943883 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:05.943891 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:05.944068 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:05.944622 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:05.944638 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:05.946700 (+ 2062us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:05.950214 (+ 3514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:05.950226 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:05.950230 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:05.950691 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:05.951135 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:05.951139 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:05.951142 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:05.951286 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:05.952048 (+ 762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:05.952053 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:05.952201 (+ 148us) client_negotiation.cc:769] Sending connection context
0809 15:13:05.952360 (+ 159us) client_negotiation.cc:240] Negotiation successful
0809 15:13:05.952581 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":273,"threads_started":1}
I20260809 15:13:05.953199 29261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.938063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46652 (local address 127.24.183.254:33563)
0809 15:13:05.939205 (+ 1142us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:05.939210 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:05.939229 (+ 19us) server_negotiation.cc:407] Connection header received
0809 15:13:05.939284 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:05.939288 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:05.939351 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:05.939445 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:05.940802 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:05.941574 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:05.943030 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:05.943650 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:05.943800 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:05.946906 (+ 3106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:05.946913 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:05.946917 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:05.946958 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:05.950094 (+ 3136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:05.950822 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:05.950828 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:05.950831 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:05.951014 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:05.951406 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:05.951412 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:05.951415 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:05.951912 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:05.952077 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:05.952496 (+ 419us) server_negotiation.cc:299] Negotiation successful
0809 15:13:05.952721 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":1024,"thread_start_us":918,"threads_started":1}
I20260809 15:13:05.954437 29252 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43017 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:13:05 dist-test-slave-nj21 krb5kdc[28926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.254@KRBTEST.COM
Aug 09 15:13:05 dist-test-slave-nj21 krb5kdc[28926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288385, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.252@KRBTEST.COM for kudu/127.24.183.253@KRBTEST.COM
I20260809 15:13:05.965862 29264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.950612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46662 (local address 127.24.183.254:33563)
0809 15:13:05.951306 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:05.951311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:05.951331 (+ 20us) server_negotiation.cc:407] Connection header received
0809 15:13:05.951369 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:05.951373 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:05.951434 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:05.951528 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:05.952995 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:05.954263 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:05.955762 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:05.956400 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:05.956528 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:05.960701 (+ 4173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:05.960706 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:05.960709 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:05.960736 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:05.963253 (+ 2517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:05.964013 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:05.964016 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:05.964019 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:05.964133 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:05.964491 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:05.964495 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:05.964497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:05.964739 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:05.964931 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:05.965277 (+ 346us) server_negotiation.cc:299] Negotiation successful
0809 15:13:05.965496 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":556,"thread_start_us":352,"threads_started":1}
I20260809 15:13:05.967283 29256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.955035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.253:43017 (local address 127.0.0.1:37968)
0809 15:13:05.955255 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:05.955271 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:05.955357 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:05.955848 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:05.955853 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:05.956274 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:05.956444 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:05.956456 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:05.957693 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:05.957697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:05.958545 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:05.958555 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:05.959472 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:05.959479 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:05.959664 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:05.960194 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:05.960210 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:05.961999 (+ 1789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:05.965118 (+ 3119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:05.965121 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:05.965123 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:05.965462 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:05.965821 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:05.965826 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:05.965829 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:05.965976 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:05.966514 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:05.966517 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:05.966628 (+ 111us) client_negotiation.cc:769] Sending connection context
0809 15:13:05.966746 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 15:13:05.966989 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127}
I20260809 15:13:05.967340 29153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.955329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37968 (local address 127.24.183.253:43017)
0809 15:13:05.955528 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:05.955535 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:05.955555 (+ 20us) server_negotiation.cc:407] Connection header received
0809 15:13:05.955613 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:05.955617 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:05.955721 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:05.955812 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:05.956587 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:05.957559 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:05.958684 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:05.959327 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:05.959420 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:05.962162 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:05.962168 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:05.962173 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:05.962213 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:05.965005 (+ 2792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:05.965575 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:05.965580 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:05.965584 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:05.965716 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:05.966079 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:05.966085 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:05.966088 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:05.966370 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:05.966529 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:05.966875 (+ 346us) server_negotiation.cc:299] Negotiation successful
0809 15:13:05.967052 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 15:13:05.968612 29252 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.183.252" port: 35287 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:13:05 dist-test-slave-nj21 krb5kdc[28926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288385, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.252@KRBTEST.COM for kudu/127.24.183.252@KRBTEST.COM
Aug 09 15:13:05 dist-test-slave-nj21 krb5kdc[28926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.183.253@KRBTEST.COM
I20260809 15:13:05.984120 29256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.969174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:35287 (local address 127.0.0.1:53664)
0809 15:13:05.969345 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:05.969362 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:05.969459 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:05.969782 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:05.969786 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:05.970227 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:05.970391 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:05.970402 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:05.971629 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:05.971632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:05.972804 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:05.972829 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:05.973656 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:05.973661 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:05.973790 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:05.974243 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:05.974259 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:05.976098 (+ 1839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:05.981950 (+ 5852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:05.981953 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:05.981956 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:05.982310 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:05.982637 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:05.982642 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:05.982646 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:05.982764 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:05.983211 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:05.983215 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:05.983338 (+ 123us) client_negotiation.cc:769] Sending connection context
0809 15:13:05.983458 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 15:13:05.983744 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 15:13:05.984262 29255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.969354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53664 (local address 127.24.183.252:35287)
0809 15:13:05.969521 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:05.969526 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:05.969544 (+ 18us) server_negotiation.cc:407] Connection header received
0809 15:13:05.969586 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:05.969591 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:05.969656 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:05.969724 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:05.970523 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:05.971503 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:05.972946 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:05.973507 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:05.973628 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:05.976266 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:05.976271 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:05.976275 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:05.976307 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:05.981821 (+ 5514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:05.982408 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:05.982411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:05.982413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:05.982536 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:05.982859 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:05.982864 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:05.982867 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:05.983107 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:05.983225 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:05.983552 (+ 327us) server_negotiation.cc:299] Negotiation successful
0809 15:13:05.983899 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":128,"server-negotiator.queue_time_us":51}
I20260809 15:13:05.984756 29153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:05.967648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37974 (local address 127.24.183.253:43017)
0809 15:13:05.968086 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:05.968090 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:05.968102 (+ 12us) server_negotiation.cc:407] Connection header received
0809 15:13:05.968142 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:05.968145 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:05.968192 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:05.968265 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:05.969622 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:05.970572 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:05.972090 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:05.972740 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:05.972862 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:05.976935 (+ 4073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:05.976939 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:05.976942 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:05.976969 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:05.981828 (+ 4859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:05.982919 (+ 1091us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:05.982923 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:05.982925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:05.983049 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:05.983393 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:05.983396 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:05.983398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:05.983647 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:05.983948 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:05.984212 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 15:13:05.984434 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344}
I20260809 15:13:05.989111 25311 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:13:05.989763 29252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1a43852e60a4bb5b1210021a219aa88: Bootstrap starting.
I20260809 15:13:05.992981 29252 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1a43852e60a4bb5b1210021a219aa88: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:05.991365 29006 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "5fdf02368961435f8ba51a79b614d954"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46662
I20260809 15:13:05.993984 29252 log.cc:824] T 00000000000000000000000000000000 P f1a43852e60a4bb5b1210021a219aa88: Log is configured to *not* fsync() on all Append() calls
I20260809 15:13:05.996605 29252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1a43852e60a4bb5b1210021a219aa88: No bootstrap required, opened a new log
I20260809 15:13:06.003356 29252 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1a43852e60a4bb5b1210021a219aa88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fdf02368961435f8ba51a79b614d954" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 33563 } } peers { permanent_uuid: "524c2e0bee4f4a5aa7ff39c8727aaff2" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43017 } } peers { permanent_uuid: "f1a43852e60a4bb5b1210021a219aa88" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 35287 } }
I20260809 15:13:06.003643 29252 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1a43852e60a4bb5b1210021a219aa88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1a43852e60a4bb5b1210021a219aa88, State: Initialized, Role: FOLLOWER
I20260809 15:13:06.004063 29252 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1a43852e60a4bb5b1210021a219aa88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fdf02368961435f8ba51a79b614d954" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 33563 } } peers { permanent_uuid: "524c2e0bee4f4a5aa7ff39c8727aaff2" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43017 } } peers { permanent_uuid: "f1a43852e60a4bb5b1210021a219aa88" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 35287 } }
I20260809 15:13:06.004918 29266 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1a43852e60a4bb5b1210021a219aa88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fdf02368961435f8ba51a79b614d954" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 33563 } } peers { permanent_uuid: "524c2e0bee4f4a5aa7ff39c8727aaff2" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43017 } } peers { permanent_uuid: "f1a43852e60a4bb5b1210021a219aa88" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 35287 } } }
I20260809 15:13:06.005288 29266 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1a43852e60a4bb5b1210021a219aa88 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:13:06.005769 29252 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1a43852e60a4bb5b1210021a219aa88 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 15:13:06.014632 29284 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f1a43852e60a4bb5b1210021a219aa88: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:13:06.014750 29284 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:13:06.020279 29041 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.252" port: 35287 } attempt: 4
I20260809 15:13:06.012471 29164 master_runner.cc:186] Error getting master registration for 127.24.183.254:33563: OK, instance_id { permanent_uuid: "5fdf02368961435f8ba51a79b614d954" instance_seqno: 1786288384884998 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 15:13:06.024024 29164 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 09 15:13:06 dist-test-slave-nj21 krb5kdc[28926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288384, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.254@KRBTEST.COM for kudu/127.24.183.252@KRBTEST.COM
I20260809 15:13:06.036374 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:06.020834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:35287 (local address 127.0.0.1:53668)
0809 15:13:06.021023 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:06.021043 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:06.021156 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:06.021508 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:06.021513 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:06.022040 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:06.022277 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:06.022292 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:06.023765 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:06.023772 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:06.025249 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:06.025265 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:06.026268 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:06.026275 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:06.026501 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:06.027136 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:06.027164 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:06.030100 (+ 2936us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:06.033930 (+ 3830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:06.033935 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:06.033940 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:06.034303 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:06.034655 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:06.034660 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:06.034665 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:06.034801 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:06.035396 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:06.035401 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:06.035558 (+ 157us) client_negotiation.cc:769] Sending connection context
0809 15:13:06.035714 (+ 156us) client_negotiation.cc:240] Negotiation successful
0809 15:13:06.036001 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 15:13:06.036381 29255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:06.021042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53668 (local address 127.24.183.252:35287)
0809 15:13:06.021203 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:06.021209 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:06.021228 (+ 19us) server_negotiation.cc:407] Connection header received
0809 15:13:06.021281 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:06.021285 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:06.021365 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:06.021457 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:06.022439 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:06.023615 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:06.025401 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:06.026112 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:06.026235 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:06.030280 (+ 4045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:06.030285 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:06.030288 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:06.030323 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:06.033805 (+ 3482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:06.034420 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:06.034426 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:06.034430 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:06.034551 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:06.034911 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:06.034917 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:06.034920 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:06.035271 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:06.035438 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:06.035819 (+ 381us) server_negotiation.cc:299] Negotiation successful
0809 15:13:06.036006 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 15:13:06.042083 29041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fdf02368961435f8ba51a79b614d954: Bootstrap starting.
I20260809 15:13:06.045370 29041 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5fdf02368961435f8ba51a79b614d954: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:06.046602 29041 log.cc:824] T 00000000000000000000000000000000 P 5fdf02368961435f8ba51a79b614d954: Log is configured to *not* fsync() on all Append() calls
I20260809 15:13:06.049944 29041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fdf02368961435f8ba51a79b614d954: No bootstrap required, opened a new log
I20260809 15:13:06.055429 29041 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5fdf02368961435f8ba51a79b614d954 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fdf02368961435f8ba51a79b614d954" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 33563 } } peers { permanent_uuid: "524c2e0bee4f4a5aa7ff39c8727aaff2" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43017 } } peers { permanent_uuid: "f1a43852e60a4bb5b1210021a219aa88" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 35287 } }
I20260809 15:13:06.055751 29041 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5fdf02368961435f8ba51a79b614d954 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fdf02368961435f8ba51a79b614d954, State: Initialized, Role: FOLLOWER
I20260809 15:13:06.056229 29041 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fdf02368961435f8ba51a79b614d954 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fdf02368961435f8ba51a79b614d954" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 33563 } } peers { permanent_uuid: "524c2e0bee4f4a5aa7ff39c8727aaff2" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43017 } } peers { permanent_uuid: "f1a43852e60a4bb5b1210021a219aa88" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 35287 } }
I20260809 15:13:06.057154 29289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fdf02368961435f8ba51a79b614d954 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fdf02368961435f8ba51a79b614d954" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 33563 } } peers { permanent_uuid: "524c2e0bee4f4a5aa7ff39c8727aaff2" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43017 } } peers { permanent_uuid: "f1a43852e60a4bb5b1210021a219aa88" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 35287 } } }
I20260809 15:13:06.057562 29289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fdf02368961435f8ba51a79b614d954 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:13:06.058314 29041 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5fdf02368961435f8ba51a79b614d954 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:13:06.058472 29006 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5fdf02368961435f8ba51a79b614d954 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:13:06.058766 29006 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5fdf02368961435f8ba51a79b614d954 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:06.062351 29006 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5fdf02368961435f8ba51a79b614d954 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fdf02368961435f8ba51a79b614d954" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 33563 } } peers { permanent_uuid: "524c2e0bee4f4a5aa7ff39c8727aaff2" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43017 } } peers { permanent_uuid: "f1a43852e60a4bb5b1210021a219aa88" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 35287 } }
I20260809 15:13:06.065536 29006 leader_election.cc:288] T 00000000000000000000000000000000 P 5fdf02368961435f8ba51a79b614d954 [CANDIDATE]: Term 1 election: Requested vote from peers 524c2e0bee4f4a5aa7ff39c8727aaff2 (127.24.183.253:43017), f1a43852e60a4bb5b1210021a219aa88 (127.24.183.252:35287)
I20260809 15:13:06.066190 29222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5fdf02368961435f8ba51a79b614d954" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f1a43852e60a4bb5b1210021a219aa88"
I20260809 15:13:06.066787 29222 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1a43852e60a4bb5b1210021a219aa88 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:06.070602 29222 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f1a43852e60a4bb5b1210021a219aa88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fdf02368961435f8ba51a79b614d954 in term 1.
I20260809 15:13:06.071256 28969 leader_election.cc:302] T 00000000000000000000000000000000 P 5fdf02368961435f8ba51a79b614d954 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fdf02368961435f8ba51a79b614d954, f1a43852e60a4bb5b1210021a219aa88; no voters:
I20260809 15:13:06.071776 29289 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5fdf02368961435f8ba51a79b614d954 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:13:06.075413 29289 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5fdf02368961435f8ba51a79b614d954 [term 1 LEADER]: Becoming Leader. State: Replica: 5fdf02368961435f8ba51a79b614d954, State: Running, Role: LEADER
I20260809 15:13:06.076089 29289 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5fdf02368961435f8ba51a79b614d954 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fdf02368961435f8ba51a79b614d954" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 33563 } } peers { permanent_uuid: "524c2e0bee4f4a5aa7ff39c8727aaff2" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43017 } } peers { permanent_uuid: "f1a43852e60a4bb5b1210021a219aa88" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 35287 } }
I20260809 15:13:06.086521 29304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fdf02368961435f8ba51a79b614d954 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5fdf02368961435f8ba51a79b614d954. Latest consensus state: current_term: 1 leader_uuid: "5fdf02368961435f8ba51a79b614d954" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fdf02368961435f8ba51a79b614d954" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 33563 } } peers { permanent_uuid: "524c2e0bee4f4a5aa7ff39c8727aaff2" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43017 } } peers { permanent_uuid: "f1a43852e60a4bb5b1210021a219aa88" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 35287 } } }
I20260809 15:13:06.086830 29304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fdf02368961435f8ba51a79b614d954 [sys.catalog]: This master's current role is: LEADER
I20260809 15:13:06.087637 29318 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:13:06.108042 29147 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.183.252" port: 35287 } attempt: 5
I20260809 15:13:06.112697 29318 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 15:13:06 dist-test-slave-nj21 krb5kdc[28926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288385, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.183.253@KRBTEST.COM for kudu/127.24.183.252@KRBTEST.COM
W20260809 15:13:06.124686 28978 tablet.cc:2369] T 00000000000000000000000000000000 P 5fdf02368961435f8ba51a79b614d954: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:13:06.126559 29151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:06.108724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.183.252:35287 (local address 127.0.0.1:53684)
0809 15:13:06.109193 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:06.109213 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:06.109333 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:06.110299 (+ 966us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:06.110305 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:06.110802 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:06.111030 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:06.111043 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:06.112613 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:06.112619 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:06.114066 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:06.114083 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:06.115401 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:06.115407 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:06.115688 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:06.116352 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:06.116377 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:06.119167 (+ 2790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:06.123639 (+ 4472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:06.123645 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:06.123649 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:06.124153 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:06.124626 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:06.124630 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:06.124634 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:06.124843 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:06.125556 (+ 713us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:06.125561 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:06.125715 (+ 154us) client_negotiation.cc:769] Sending connection context
0809 15:13:06.125859 (+ 144us) client_negotiation.cc:240] Negotiation successful
0809 15:13:06.126158 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345}
I20260809 15:13:06.127645 29255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:06.109609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53684 (local address 127.24.183.252:35287)
0809 15:13:06.109796 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:06.109802 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:06.109820 (+ 18us) server_negotiation.cc:407] Connection header received
0809 15:13:06.109886 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:06.109892 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:06.109992 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:06.110086 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:06.111212 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:06.112444 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:06.114347 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:06.115064 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:06.115183 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:06.119417 (+ 4234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:06.119424 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:06.119429 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:06.119472 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:06.123490 (+ 4018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:06.124297 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:06.124303 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:06.124307 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:06.124486 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:06.124986 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:06.124991 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:06.124994 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:06.125350 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:06.125493 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:06.126912 (+ 1419us) server_negotiation.cc:299] Negotiation successful
0809 15:13:06.127147 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 15:13:06.133527 29147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 524c2e0bee4f4a5aa7ff39c8727aaff2: Bootstrap starting.
I20260809 15:13:06.137720 29147 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 524c2e0bee4f4a5aa7ff39c8727aaff2: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:06.139129 29147 log.cc:824] T 00000000000000000000000000000000 P 524c2e0bee4f4a5aa7ff39c8727aaff2: Log is configured to *not* fsync() on all Append() calls
I20260809 15:13:06.120296 28942 master_runner.cc:186] Error getting master registration for 127.24.183.253:43017: OK, instance_id { permanent_uuid: "524c2e0bee4f4a5aa7ff39c8727aaff2" instance_seqno: 1786288385387595 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 15:13:06.149201 28942 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 15:13:06.154816 29192 tablet.cc:2369] T 00000000000000000000000000000000 P f1a43852e60a4bb5b1210021a219aa88: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:13:06.159756 29264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:06.093282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46666 (local address 127.24.183.254:33563)
0809 15:13:06.093843 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:06.093851 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:06.103185 (+ 9334us) server_negotiation.cc:407] Connection header received
0809 15:13:06.104653 (+ 1468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:06.104660 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:06.104751 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:06.104827 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:06.119553 (+ 14726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:06.120978 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:06.134790 (+ 13812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:06.135516 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:06.135627 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:06.148494 (+ 12867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:06.148504 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:06.148509 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:06.148561 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:06.154341 (+ 5780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:06.157092 (+ 2751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:06.157099 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:06.157103 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:06.157307 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:06.157766 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:06.157773 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:06.157776 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:06.158194 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:06.158502 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:06.159085 (+ 583us) server_negotiation.cc:299] Negotiation successful
0809 15:13:06.159358 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386}
I20260809 15:13:06.161106 29222 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f1a43852e60a4bb5b1210021a219aa88 [term 1 FOLLOWER]: Refusing update from remote peer 5fdf02368961435f8ba51a79b614d954: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:13:06.162668 29304 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5fdf02368961435f8ba51a79b614d954 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1a43852e60a4bb5b1210021a219aa88" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 35287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:13:06.166512 29147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 524c2e0bee4f4a5aa7ff39c8727aaff2: No bootstrap required, opened a new log
I20260809 15:13:06.174338 29330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:06.124742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53696 (local address 127.24.183.252:35287)
0809 15:13:06.132102 (+ 7360us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:06.132107 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:06.134405 (+ 2298us) server_negotiation.cc:407] Connection header received
0809 15:13:06.134563 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:06.134568 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:06.134638 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:06.134732 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:06.149641 (+ 14909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:06.151002 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:06.152944 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:06.153605 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:06.153709 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:06.157350 (+ 3641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:06.157358 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:06.157363 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:06.157403 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:06.169299 (+ 11896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:06.170163 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:06.170169 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:06.170173 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:06.170328 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:06.171336 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:06.171342 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:06.171346 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:06.171838 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:06.172098 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:06.173406 (+ 1308us) server_negotiation.cc:299] Negotiation successful
0809 15:13:06.173640 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7242,"thread_start_us":7089,"threads_started":1}
I20260809 15:13:06.183699 29153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:06.103825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37980 (local address 127.24.183.253:43017)
0809 15:13:06.104020 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:06.104026 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:06.124252 (+ 20226us) server_negotiation.cc:407] Connection header received
0809 15:13:06.132216 (+ 7964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:06.132222 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:06.132314 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:06.132380 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:06.141892 (+ 9512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:06.143163 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:06.150052 (+ 6889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:06.150761 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:06.150860 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:06.173220 (+ 22360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:06.173226 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:06.173230 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:06.173285 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:06.176948 (+ 3663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:06.179471 (+ 2523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:06.179478 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:06.179481 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:06.179653 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:06.180870 (+ 1217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:06.180876 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:06.180880 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:06.181264 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:06.181508 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:06.183033 (+ 1525us) server_negotiation.cc:299] Negotiation successful
0809 15:13:06.183300 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 15:13:06.193245 29266 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1a43852e60a4bb5b1210021a219aa88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5fdf02368961435f8ba51a79b614d954. Latest consensus state: current_term: 1 leader_uuid: "5fdf02368961435f8ba51a79b614d954" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fdf02368961435f8ba51a79b614d954" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 33563 } } peers { permanent_uuid: "524c2e0bee4f4a5aa7ff39c8727aaff2" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43017 } } peers { permanent_uuid: "f1a43852e60a4bb5b1210021a219aa88" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 35287 } } }
I20260809 15:13:06.193586 29266 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1a43852e60a4bb5b1210021a219aa88 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:13:06.196139 29147 raft_consensus.cc:348] T 00000000000000000000000000000000 P 524c2e0bee4f4a5aa7ff39c8727aaff2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fdf02368961435f8ba51a79b614d954" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 33563 } } peers { permanent_uuid: "524c2e0bee4f4a5aa7ff39c8727aaff2" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43017 } } peers { permanent_uuid: "f1a43852e60a4bb5b1210021a219aa88" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 35287 } }
I20260809 15:13:06.196929 29147 raft_consensus.cc:729] T 00000000000000000000000000000000 P 524c2e0bee4f4a5aa7ff39c8727aaff2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 524c2e0bee4f4a5aa7ff39c8727aaff2, State: Initialized, Role: FOLLOWER
I20260809 15:13:06.197898 29332 mvcc.cc:205] Tried to move back new op lower bound from 7316637229699833856 to 7316637229374349312. Current Snapshot: MvccSnapshot[applied={T|T < 7316637229699833856}]
I20260809 15:13:06.197815 29147 consensus_queue.cc:260] T 00000000000000000000000000000000 P 524c2e0bee4f4a5aa7ff39c8727aaff2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fdf02368961435f8ba51a79b614d954" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 33563 } } peers { permanent_uuid: "524c2e0bee4f4a5aa7ff39c8727aaff2" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43017 } } peers { permanent_uuid: "f1a43852e60a4bb5b1210021a219aa88" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 35287 } }
I20260809 15:13:06.198926 29304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fdf02368961435f8ba51a79b614d954 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5fdf02368961435f8ba51a79b614d954" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fdf02368961435f8ba51a79b614d954" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 33563 } } peers { permanent_uuid: "524c2e0bee4f4a5aa7ff39c8727aaff2" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43017 } } peers { permanent_uuid: "f1a43852e60a4bb5b1210021a219aa88" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 35287 } } }
I20260809 15:13:06.199265 29304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fdf02368961435f8ba51a79b614d954 [sys.catalog]: This master's current role is: LEADER
I20260809 15:13:06.202555 29318 catalog_manager.cc:1347] Generated new cluster ID: ccceb924dd7245309ca0349be4b32989
I20260809 15:13:06.202658 29318 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:13:06.211441 29339 sys_catalog.cc:455] T 00000000000000000000000000000000 P 524c2e0bee4f4a5aa7ff39c8727aaff2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fdf02368961435f8ba51a79b614d954" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 33563 } } peers { permanent_uuid: "524c2e0bee4f4a5aa7ff39c8727aaff2" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43017 } } peers { permanent_uuid: "f1a43852e60a4bb5b1210021a219aa88" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 35287 } } }
I20260809 15:13:06.212081 29339 sys_catalog.cc:458] T 00000000000000000000000000000000 P 524c2e0bee4f4a5aa7ff39c8727aaff2 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:13:06.212088 29266 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1a43852e60a4bb5b1210021a219aa88 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5fdf02368961435f8ba51a79b614d954" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fdf02368961435f8ba51a79b614d954" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 33563 } } peers { permanent_uuid: "524c2e0bee4f4a5aa7ff39c8727aaff2" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43017 } } peers { permanent_uuid: "f1a43852e60a4bb5b1210021a219aa88" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 35287 } } }
I20260809 15:13:06.212400 29266 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1a43852e60a4bb5b1210021a219aa88 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:13:06.214329 29108 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 524c2e0bee4f4a5aa7ff39c8727aaff2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:06.214869 29109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5fdf02368961435f8ba51a79b614d954" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "524c2e0bee4f4a5aa7ff39c8727aaff2"
I20260809 15:13:06.218166 29108 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 524c2e0bee4f4a5aa7ff39c8727aaff2 [term 1 FOLLOWER]: Refusing update from remote peer 5fdf02368961435f8ba51a79b614d954: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:13:06.218786 29147 sys_catalog.cc:564] T 00000000000000000000000000000000 P 524c2e0bee4f4a5aa7ff39c8727aaff2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:13:06.219877 29304 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5fdf02368961435f8ba51a79b614d954 [LEADER]: Connected to new peer: Peer: permanent_uuid: "524c2e0bee4f4a5aa7ff39c8727aaff2" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:13:06.221762 29109 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 524c2e0bee4f4a5aa7ff39c8727aaff2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fdf02368961435f8ba51a79b614d954 in term 1.
I20260809 15:13:06.247217 29318 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:13:06.249123 29318 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:13:06.272706 29318 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5fdf02368961435f8ba51a79b614d954: Generated new TSK 0
I20260809 15:13:06.273907 29318 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:13:06.276711 29339 sys_catalog.cc:455] T 00000000000000000000000000000000 P 524c2e0bee4f4a5aa7ff39c8727aaff2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5fdf02368961435f8ba51a79b614d954. Latest consensus state: current_term: 1 leader_uuid: "5fdf02368961435f8ba51a79b614d954" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fdf02368961435f8ba51a79b614d954" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 33563 } } peers { permanent_uuid: "524c2e0bee4f4a5aa7ff39c8727aaff2" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43017 } } peers { permanent_uuid: "f1a43852e60a4bb5b1210021a219aa88" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 35287 } } }
I20260809 15:13:06.277015 29339 sys_catalog.cc:458] T 00000000000000000000000000000000 P 524c2e0bee4f4a5aa7ff39c8727aaff2 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:13:06.277454 29346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 524c2e0bee4f4a5aa7ff39c8727aaff2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5fdf02368961435f8ba51a79b614d954" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fdf02368961435f8ba51a79b614d954" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 33563 } } peers { permanent_uuid: "524c2e0bee4f4a5aa7ff39c8727aaff2" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43017 } } peers { permanent_uuid: "f1a43852e60a4bb5b1210021a219aa88" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 35287 } } }
I20260809 15:13:06.277880 29346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 524c2e0bee4f4a5aa7ff39c8727aaff2 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:13:06.277953 29304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fdf02368961435f8ba51a79b614d954 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5fdf02368961435f8ba51a79b614d954" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fdf02368961435f8ba51a79b614d954" member_type: VOTER last_known_addr { host: "127.24.183.254" port: 33563 } } peers { permanent_uuid: "524c2e0bee4f4a5aa7ff39c8727aaff2" member_type: VOTER last_known_addr { host: "127.24.183.253" port: 43017 } } peers { permanent_uuid: "f1a43852e60a4bb5b1210021a219aa88" member_type: VOTER last_known_addr { host: "127.24.183.252" port: 35287 } } }
I20260809 15:13:06.278185 29304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fdf02368961435f8ba51a79b614d954 [sys.catalog]: This master's current role is: LEADER
I20260809 15:13:06.367660 29364 catalog_manager.cc:1259] Loaded cluster ID: ccceb924dd7245309ca0349be4b32989
I20260809 15:13:06.367816 29364 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 524c2e0bee4f4a5aa7ff39c8727aaff2: loading cluster ID for follower catalog manager: success
I20260809 15:13:06.371613 29364 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 524c2e0bee4f4a5aa7ff39c8727aaff2: acquiring CA information for follower catalog manager: success
I20260809 15:13:06.374097 29364 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 524c2e0bee4f4a5aa7ff39c8727aaff2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 15:13:06.482435 29264 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:13:06.469330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46682 (local address 127.24.183.254:33563)
0809 15:13:06.469536 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:06.469543 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:06.480305 (+ 10762us) server_negotiation.cc:407] Connection header received
0809 15:13:06.480612 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:06.480619 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:06.480711 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:06.480788 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:06.482132 (+ 1344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46682: BlockingRecv error: recv got EOF from 127.0.0.1:46682 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260809 15:13:06.484520 29153 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:13:06.480520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37994 (local address 127.24.183.253:43017)
0809 15:13:06.480756 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:06.480762 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:06.482952 (+ 2190us) server_negotiation.cc:407] Connection header received
0809 15:13:06.483089 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:06.483095 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:06.483202 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:06.483305 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:06.484275 (+ 970us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37994: BlockingRecv error: recv got EOF from 127.0.0.1:37994 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260809 15:13:06.492986 29330 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:13:06.490256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53702 (local address 127.24.183.252:35287)
0809 15:13:06.490453 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:06.490459 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:06.491157 (+ 698us) server_negotiation.cc:407] Connection header received
0809 15:13:06.491407 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:06.491414 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:06.491520 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:06.491591 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:06.492696 (+ 1105us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53702: BlockingRecv error: recv got EOF from 127.0.0.1:53702 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 15:13:06.544302 29153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:06.521344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38010 (local address 127.24.183.253:43017)
0809 15:13:06.521558 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:06.521565 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:06.536269 (+ 14704us) server_negotiation.cc:407] Connection header received
0809 15:13:06.536528 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:06.536535 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:06.536809 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:06.536982 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:13:06.537731 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:06.538937 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:06.541087 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:06.541778 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:06.541880 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:06.542659 (+ 779us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:13:06.543395 (+ 736us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:13:06.543512 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:06.543811 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 15:13:06.543925 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260809 15:13:06.590390 29330 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:13:06.574146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53708 (local address 127.24.183.252:35287)
0809 15:13:06.574374 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:06.574381 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:06.588346 (+ 13965us) server_negotiation.cc:407] Connection header received
0809 15:13:06.588624 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:06.588632 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:06.588809 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:06.588895 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:06.590113 (+ 1218us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53708: BlockingRecv error: recv got EOF from 127.0.0.1:53708 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 15:13:06.639266 29153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:06.617829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38026 (local address 127.24.183.253:43017)
0809 15:13:06.618031 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:06.618038 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:06.632388 (+ 14350us) server_negotiation.cc:407] Connection header received
0809 15:13:06.632661 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:06.632668 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:06.632779 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:06.632852 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:13:06.633597 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:06.634764 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:06.636854 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:06.637527 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:06.637632 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:06.638328 (+ 696us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:13:06.638511 (+ 183us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:13:06.638577 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:06.638836 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 15:13:06.638933 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260809 15:13:06.663666 29330 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:13:06.661315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53722 (local address 127.24.183.252:35287)
0809 15:13:06.661505 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:06.661513 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:06.662048 (+ 535us) server_negotiation.cc:407] Connection header received
0809 15:13:06.662247 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:06.662253 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:06.662358 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:06.662472 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:06.663424 (+ 952us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53722: BlockingRecv error: recv got EOF from 127.0.0.1:53722 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 15:13:06.676888 29153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:06.669554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38032 (local address 127.24.183.253:43017)
0809 15:13:06.669772 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:06.669778 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:06.670291 (+ 513us) server_negotiation.cc:407] Connection header received
0809 15:13:06.670501 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:06.670507 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:06.670609 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:06.670721 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:13:06.671216 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:06.672483 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:06.674433 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:06.675172 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:06.675314 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:06.675914 (+ 600us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:13:06.676058 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:13:06.676151 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:06.676388 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 15:13:06.676485 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260809 15:13:06.685304 29330 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:13:06.683068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53728 (local address 127.24.183.252:35287)
0809 15:13:06.683263 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:06.683269 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:06.683829 (+ 560us) server_negotiation.cc:407] Connection header received
0809 15:13:06.683982 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:06.683987 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:06.684081 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:06.684175 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:06.685074 (+ 899us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53728: BlockingRecv error: recv got EOF from 127.0.0.1:53728 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 15:13:06.699561 29153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:06.692421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38048 (local address 127.24.183.253:43017)
0809 15:13:06.692616 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:06.692622 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:06.693106 (+ 484us) server_negotiation.cc:407] Connection header received
0809 15:13:06.693407 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:06.693416 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:06.693513 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:06.693578 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:13:06.694236 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:06.695445 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:06.697478 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:06.698183 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:06.698320 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:06.698805 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:13:06.698943 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:13:06.699026 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:06.699191 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 15:13:06.699284 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260809 15:13:06.716493 29330 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:13:06.713948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53738 (local address 127.24.183.252:35287)
0809 15:13:06.714222 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:06.714229 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:06.714830 (+ 601us) server_negotiation.cc:407] Connection header received
0809 15:13:06.715028 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:06.715034 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:06.715133 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:06.715247 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:06.716243 (+ 996us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53738: BlockingRecv error: recv got EOF from 127.0.0.1:53738 (error 108)
Metrics: {"server-negotiator.queue_time_us":132}
I20260809 15:13:06.735507 29153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:06.729547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38058 (local address 127.24.183.253:43017)
0809 15:13:06.729733 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:06.729739 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:06.730705 (+ 966us) server_negotiation.cc:407] Connection header received
0809 15:13:06.730908 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:06.730914 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:06.730992 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:06.731069 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:13:06.731471 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:06.732306 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:06.733675 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:06.734196 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:06.734327 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:06.734754 (+ 427us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:13:06.734900 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:13:06.734977 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:06.735116 (+ 139us) server_negotiation.cc:299] Negotiation successful
0809 15:13:06.735205 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260809 15:13:06.742990 29330 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:13:06.741092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53744 (local address 127.24.183.252:35287)
0809 15:13:06.741268 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:06.741274 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:06.741646 (+ 372us) server_negotiation.cc:407] Connection header received
0809 15:13:06.741812 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:06.741817 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:06.741896 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:06.741979 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:06.742729 (+ 750us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53744: BlockingRecv error: recv got EOF from 127.0.0.1:53744 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 15:13:06.770224 29153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:06.763580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38074 (local address 127.24.183.253:43017)
0809 15:13:06.763794 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:06.763800 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:06.764373 (+ 573us) server_negotiation.cc:407] Connection header received
0809 15:13:06.764648 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:06.764653 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:06.764790 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:06.764871 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:13:06.765384 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:06.766216 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:06.768337 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:06.768864 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:06.768974 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:06.769519 (+ 545us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:13:06.769642 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:13:06.769712 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:06.769894 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 15:13:06.769975 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260809 15:13:06.778872 29330 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:13:06.776735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53756 (local address 127.24.183.252:35287)
0809 15:13:06.776927 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:06.776934 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:06.777386 (+ 452us) server_negotiation.cc:407] Connection header received
0809 15:13:06.777552 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:06.777558 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:06.777662 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:06.777785 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:06.778664 (+ 879us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53756: BlockingRecv error: recv got EOF from 127.0.0.1:53756 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 15:13:06.821270 29153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:06.814694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38082 (local address 127.24.183.253:43017)
0809 15:13:06.814977 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:06.814985 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:06.815310 (+ 325us) server_negotiation.cc:407] Connection header received
0809 15:13:06.815504 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:06.815511 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:06.815655 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:06.815827 (+ 172us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:13:06.816256 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:06.817482 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:06.818820 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:06.819656 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:06.819832 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:06.820321 (+ 489us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:13:06.820490 (+ 169us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:13:06.820613 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:06.820801 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 15:13:06.820908 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
W20260809 15:13:06.829481 29330 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:13:06.827458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53758 (local address 127.24.183.252:35287)
0809 15:13:06.827692 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:06.827701 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:06.827993 (+ 292us) server_negotiation.cc:407] Connection header received
0809 15:13:06.828162 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:06.828170 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:06.828305 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:06.828419 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:06.829170 (+ 751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53758: BlockingRecv error: recv got EOF from 127.0.0.1:53758 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 15:13:06.902685 29153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:06.896962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38094 (local address 127.24.183.253:43017)
0809 15:13:06.897166 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:06.897172 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:06.897513 (+ 341us) server_negotiation.cc:407] Connection header received
0809 15:13:06.897720 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:06.897724 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:06.897801 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:06.897893 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:13:06.898420 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:06.899144 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:06.901136 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:06.901552 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:06.901641 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:06.902056 (+ 415us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:13:06.902159 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:13:06.902218 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:06.902360 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 15:13:06.902428 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260809 15:13:06.910490 29330 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:13:06.908491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53760 (local address 127.24.183.252:35287)
0809 15:13:06.908668 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:06.908674 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:06.909119 (+ 445us) server_negotiation.cc:407] Connection header received
0809 15:13:06.909305 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:06.909310 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:06.909396 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:06.909493 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:06.910263 (+ 770us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53760: BlockingRecv error: recv got EOF from 127.0.0.1:53760 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 15:13:07.016891 29284 catalog_manager.cc:1259] Loaded cluster ID: ccceb924dd7245309ca0349be4b32989
I20260809 15:13:07.016991 29284 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f1a43852e60a4bb5b1210021a219aa88: loading cluster ID for follower catalog manager: success
I20260809 15:13:07.019585 29284 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f1a43852e60a4bb5b1210021a219aa88: acquiring CA information for follower catalog manager: success
I20260809 15:13:07.021489 29284 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f1a43852e60a4bb5b1210021a219aa88: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 15:13:07.046741 29153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:07.041863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38108 (local address 127.24.183.253:43017)
0809 15:13:07.042125 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:07.042132 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:07.042398 (+ 266us) server_negotiation.cc:407] Connection header received
0809 15:13:07.042543 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:07.042547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:07.042636 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:07.042734 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:13:07.043141 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:07.043918 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:07.045205 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:07.045647 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:07.045745 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:07.046107 (+ 362us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:13:07.046227 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:13:07.046301 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:07.046392 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 15:13:07.046460 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260809 15:13:07.057637 29330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:07.052042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53770 (local address 127.24.183.252:35287)
0809 15:13:07.052246 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:07.052253 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:07.052495 (+ 242us) server_negotiation.cc:407] Connection header received
0809 15:13:07.052686 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:07.052690 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:07.052855 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:07.052960 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:13:07.053359 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:07.054185 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:07.055476 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:07.055920 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:07.056033 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:07.056522 (+ 489us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:13:07.057036 (+ 514us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:13:07.057170 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:07.057272 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 15:13:07.057371 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 15:13:07.069393 29524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:07.063177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46684 (local address 127.24.183.254:33563)
0809 15:13:07.063694 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:07.063701 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:07.063724 (+ 23us) server_negotiation.cc:407] Connection header received
0809 15:13:07.063980 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:07.063985 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:07.064205 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:07.064395 (+ 190us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:13:07.064815 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:07.065660 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:07.067016 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:07.067410 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:07.067520 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:07.068130 (+ 610us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:13:07.068632 (+ 502us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:13:07.068755 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:07.068940 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 15:13:07.069040 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":248,"threads_started":1}
I20260809 15:13:07.074012 29153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:07.063970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38112 (local address 127.24.183.253:43017)
0809 15:13:07.064175 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:07.064181 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:07.068184 (+ 4003us) server_negotiation.cc:407] Connection header received
0809 15:13:07.068458 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:07.068465 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:07.068568 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:07.068622 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:13:07.069350 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:07.070395 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:07.072025 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:07.072614 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:07.072741 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:07.073266 (+ 525us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:13:07.073402 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:13:07.073491 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:07.073644 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 15:13:07.073731 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 15:13:07.075064 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 28942
I20260809 15:13:07.075273 29330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:07.068322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53772 (local address 127.24.183.252:35287)
0809 15:13:07.068501 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:07.068506 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:07.069261 (+ 755us) server_negotiation.cc:407] Connection header received
0809 15:13:07.069526 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:07.069531 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:07.069614 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:07.069673 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:13:07.070326 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:07.071230 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:07.073302 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:07.073903 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:07.074024 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:07.074458 (+ 434us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:13:07.074602 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:13:07.074686 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:07.074893 (+ 207us) server_negotiation.cc:299] Negotiation successful
0809 15:13:07.075003 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 15:13:07.076095 29036 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:13:07.216521 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 28942
I20260809 15:13:07.252138 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 29051
I20260809 15:13:07.253268 29141 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:13:07.363976 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 29051
I20260809 15:13:07.388401 25311 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu pid 29164
I20260809 15:13:07.389537 29246 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:13:07.499517 25311 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXnZFlc/build/asan/bin/kudu with pid 29164
2026-08-09T15:13:07Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4594 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9678 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (366150 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST